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FOREWORD 



I n 2002, when the Council for Aid to Education (CAE) first asked the Council of Independent Colleges (CIC) to help 
select a few colleges to participate in a pilot project that would measure the effects of an institution on how much the 
students had learned during college, CIC jumped at the opportunity. It had been our view that the prevailing and largely 
anecdotal ways of describing the distinctive educational advantages of smaller, largely residential, liberal arts-based, private 
colleges and universities had been only modestly persuasive and a more empirical approach was needed. The Collegiate 
beaming Assessment appeared to be well suited to the challenge we faced. It avoided the dangers of “high stakes” tests that 
could intimidate individual students and faculty members by keeping its focus on the institution as a whole. It did not require 
large amounts of time to administer. And — a particular advantage for small colleges — it did not require large amounts of 
money or specialized staff expertise in order to be used effectively. 

The response by CIC colleges and universities was very positive and, over time, increasing numbers of CIC colleges made 
commitments to administer the CEA and to use the results to stimulate reforms in teaching and learning on campus. 
Between 2002 and 2011 several foundations — chief among them the Teagle Eoundation — supported efforts to develop and 
expand a consortium of CIC colleges that had pledged to use the CEA and to share results with one another. Over the years, 
our hopes of identifying effective practices that actually improve how much students learn between their freshman and 
senior years were largely fulfilled. 

CIC’s interest in this venture was — and remains — a reflection of the biggest story in American higher education during the 
second half of the 20th century: the massive expansion of college-going to include two-thirds of all high school graduates, 
including those with middling academic abilities and those from “at risk” groups of the population who had previously been 
underrepresented in higher education. These groups include first-generation, low-income, and minority students. How were 
the rank and file of American colleges doing, we asked, in educating these new college-goers? Did a medium-selective private 
college with a commitment to enrolling and graduating students with these characteristics do a good job in advancing its 
students’ cognitive development over the four years of college? Because there was some evidence to suggest that, on average, 
our colleges are more effective than other kinds of institutions for students with these characteristics, the CEA offered a 
promising means to document the performance of independent institutions. 

CIC has long been a national leader in voluntary efforts to improve the quality of student learning and a strong advocate 
of institutional autonomy in accountability efforts. Eor more than a decade, CIC has tried to help its member colleges 
navigate through the array of initiatives that are underway — those by regional accreditors, foundation-supported projects, 
and various campus efforts — focusing on specific aspects of the overall topic at different times. In 2001, CIC was the first 
national presidential association to urge its members to use the National Survey of Student Engagement (NSSE) as a means 
of gauging student involvement in educationally purposeful activities that are highly correlated with academic success. In 
2004, well before then-Secretary of Education Margaret Spellings’ Commission on the Euture of Higher Education proposed 
to mandate the use of outcomes assessment, CIC had established its CEA Consortium. 

Now, nearly a decade later, more than four dozen CIC institutions have been part of this voluntary effort. In the interim, 
regional accrediting bodies, state agencies, and more than one federal official have called for mandatory assessment 
of students’ “learning outcomes” and some state systems have provided funding to campuses to administer the CEA. 
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Nonetheless, the voluntary CIC Consortium is still the 
largest group of colleges that work together to learn from 
one another’s experiences in the use of the CLA. 

In 2008 we produced a progress report on the Consortium’s 
work that stimulated other colleges and universities — 
public and private — to launch their own projects based 
on use of the CLA. Now CIC has produced a final report 
on the Consortium’s work — final, not in the sense that 
the colleges are dropping their use of the CLA (most are 
continuing), but final in that the annual Consortium 
meetings, supported by the Teagle Foundation, are ending 
after seven years of very generous funding. Both CIC 
and the Teagle Foundation are willing to declare victory: 
the use of the CLA has been piloted in a wide variety of 
institutions; many colleges and universities have developed 
habits of using it to propel reform efforts in teaching 
and learning practices; the collaborative approach of a 
consortium of institutions has proven to be a catalyst for 
change on individual campuses and with this report seeks 
to stimulate similar change in other institutions; and 
the hunch in 2002 that voluntary efforts will work and 
government mandates are not necessary appears to have 
been amply demonstrated. 

CIC owes a great deal to the Teagle Foundation’s stalwart support and to its past and current presidents. Bob Connor and 
Richard Morrill, and its vice president, Donna Heiland. We also thank our CAE colleagues Roger Benjamin, president, 
and Marc Chun, director of education. Guiding this work at CIC from the start, with both deft diplomacy and excellent 
judgment, has been Hal Hartley, senior vice president. The informative report that follows was written largely by David Paris, 
a CIC senior advisor and now executive director of the New Leadership Alliance for Student Learning and Accountability. 

I hope that readers of this report will find in it helpful advice that enables them to advance efforts on their own campuses to 
assess students’ cognitive growth. 

Richard Ekman 
President 

Council of Independent Colleges 
Washington, DC 

October 2011 




Alaska Pacific University, established in 1959, is a small 
independent university devoted to innovative teaching and 
learning, where students receive personal attention in small 
classes and acquire leadership capabilities, moral character, 
and self-direction through active learning. To achieve these 
educational goals, the university nurtures spiritual and 
moral values consistent with the its Christian heritage while 
respecting the religious convictions of all. The university now 
offers 11 undergraduate majors and five graduate programs. 
www.alaskapacific.edu 
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EXECUTIVE SUMMARY 



F rom the fall of 2008 through the spring of 2011, 

47 colleges and universities, organized and supported by 
the Council of Independent Colleges (CIC), administered 
the Collegiate Learning Assessment (CLA) to their 
students. The CLA is a test of critical thinking, analytic 
reasoning, problem solving, and written communication 
developed by the Council for Aid to Education. The purpose 
of the CIC/CLA Consortium, underwritten by grants 
from the Teagle Foundation, was to embed a “culture of 
assessment” on participating campuses, refine the methods 
used to assess student learning, and identify “best practices” 
for the improvement of student learning and teaching that 
could be confidently shared with other small and mid-sized 
independent colleges and universities and made known 
to the public. Each institution was free to develop its own 
way of administering and using the CLA. In this last phase 
of the Consortium, summer meetings were held in 2007 
through 2011 during which institutions compared notes, 
tactics, and results as well as discussed strategies to improve 
student learning on their campuses. 

This monograph provides a summary of the experiences 
of Consortium members. Participating in the Consortium 
led to a wide variety of effects. These ranged from fairly 
immediate changes in program and pedagogy to indirect, 
but no less important, shifts in conversations among faculty 
members and with administrators and in the campus 
culture. The initiative also led to increased interest in 
teaching critical thinking through the presentation of ill- 
structured problems, particularly CLAs “performance tasks.” 
With each annual iteration, institutions steadily developed 
better ways of administering and using the CLA and 
conducting other assessment exercises. Overall, participation 
in the Consortium had a significant, positive impact on the 
vast majority of the institutions. The initiative produced a 
steady expansion of efforts, changes, experimentation, and 
conversation. 

One of the words most frequently used in the CIC/CLA 
Consortium campus reports is “catalyst.” The CLA results 
might not be immediately or directly connected to a change 
in program or pedagogy in every case, but the instrument 



did spark consideration of what assessment should be 
and do on a campus and which academic programs or 
pedagogies need to be revisited. These “catalytic” effects 
involved both programmatic and pedagogical effects 
as well as changes in institutional culture. Sometimes 
programmatic effects included the CLA influencing how 
assessment activities occurred on campus. The CLA served 
as a catalyst for rethinking assessment among Consortium 
colleges and involved the triangulation of CLA results with 
other measures. The CLA was also a catalyst for changes 
in academic programs and pedagogy. The administration 
of the CLA prompted a reexamination of programs and 
practices that, though not always directly tied to the CLA 
instrument or its results, are nonetheless a significant 
change in an institution’s approach to student learning. 
These changes varied from the revamping of a broad, 
perhaps institution-wide assessment program to shifts in 
programs and pedagogy. 

The experience and success of the CIC/CLA 
Consortium offer lessons for other colleges and universities 
as well as policy officials, as they respond to demands for 
more assessment of, and accountability for, student learning 
outcomes. 

The first lesson is that measures and measurement 
matter. The CLA is a powerful and potentially valuable 
instrument, and, equally important, having a key measure 
or measures is crucial to focusing discussions of student 
learning within an institution. Every institution wants to 
improve its students’ critical thinking skills, and the CLA 
provides an opportunity to measure those skills in ways 
that most faculty members recognize as being important 
and authentic. Similarly, once a measure like the CLA is 
used it establishes the principle that an institution can and 
should consider it as well as other types of measures and 
evidence. The frequent attempts to triangulate CLA results 
with National Survey of Student Engagement (NSSE) 
and to conduct analyses of CLA by area or program reflect 
the fact that there are a variety of measures available 
and analyses that can be done. Whatever other changes 
have occurred at the colleges in the Consortium, what 
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has changed dramatically is a disposition toward use of 
evidence and analysis of student learning. 

Second, and similarly, the process of measuring student 
learning creates a catalyst for institutional change — and 
it is difficult to imagine reversing this momentum. As 
most of the institutions in the Consortium reported, 
once begun, assessment becomes part of the conversation 
and an ongoing activity of the institution. For some 
institutions the reaction was immediate and moved to 
focusing on those areas — especially critical thinking and 
writing — assessed by the CLA. These and other efforts 
have expanded in a range of ways, from changing specific 
courses and programs, to focusing faculty development 
efforts, to revisiting how assessment was conducted on 
campus more generally. Once these changes are made, the 
logic of assessment and working to “close the loop” develop 
a momentum of their own. 

A third lesson is that assessment and institutional 
change can be greatly aided by collaboration among 
institutions. The CIC/CLA Consortium illustrates 
the possibility of a professional community of practice 
that supports assessment and improvement of student 
learning. Common measures and common issues across 
institutions give the assessment process some measure of 
credibility. Having other institutions provide advice on 
everything from logistical challenges to the broadest ideas 
about curriculum and program creates a community of 
professional practice that makes it easier to improve an 
institution’s work. The work of the CIC/CLA Consortium 
provides a model of how undergraduate education 
can become more professionalized through shared 
understandings, measures, and practices. 

A final lesson that can be drawn from the CIC initiative 
emerges from the recognition that the pressure on higher 
education to demonstrate its value, especially in light of 
rising costs, is not likely to fade. Worse, the temptation 
for public officials to use regulation as a means to spur 




Allegheny College (Pennsylvania), established in 1815, is the 
32nd oldest college in the nation and will celebrate its bi- 
centennial in 2015. One of the 40 institutions featured in Loren 
Pope's Colleges That Change Lives, 90 percent of Allegheny 
alumni seeking employment start a career within eight months 
of graduation, and the college ranks in the top 5 percent 
of colleges nationally whose students go on to earn PhDs. 
Allegheny enrolls more than 2,100 students, www.allegheny.edu 



innovation and improvement is also likely to be present, 
at least for the near term. The CIC/CLA Consortium 
offers an example of self-directed, voluntary professional 
efforts that have and will continue to provide a sound 
response to legitimate public demands for assessment, 
accountability, and improvement. The distinctive system 
of higher education in the United States — its mix of 
diverse, autonomous institutions, including a robust 
private sector — once engaged in thoughtful cooperation 
and experimentation, does not need external direction 
to continue to improve its work with students. What 
is needed, and found in the CIC/CLA Consortium, is 
the dedicated work of individuals and institutions in 
higher education, particularly in the independent sector, 
constantly working together to improve the education of 
their students. ■ 
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I. WHO’S LEARNING. ..AND HOW DO YOU KNOW?: THE COUNCIL OF 
INDEPENDENT COLLEGES AND THE COLLEGIATE LEARNING ASSESSMENT 



There is no simple blueprint, roadmap, or formula for establishing a 
successful assessment program, with or without the CLA. This endeavor 
provided the opportunity for making improvements, especially as 
institutions drew from the experience of colleagues through the CIC/CLA 
Consortium. 



O ver the past three decades colleges and universities 
have improved their efforts to assess student 
learning. There are a number of reasons for this — a sense 
of professional responsibility, state and accreditation 
agency reporting requirements, and greater clarity and 
understanding about the process and measurement 
of learning. Public officials and the general public 
have also called for greater transparency about, and 
accountability for, what students are actually achieving. 
Given the importance of higher education for economic 
competitiveness and civic participation and the large 
investment of public, philanthropic, and family resources in 
higher education, the recent demands for better assessment 
and disclosure of results in assessment of student learning 
are probably inevitable and surely reasonable. 

The 2006 Department of Education report, A Test of 
Leadership: Charting the Future of U.S. Higher Education, 
commonly known as the Spellings Commission report, 
increased pressure for greater accountability. That 
report suggested that the traditional advantage of the 
United States in higher education was eroding as greater 
percentages of students in other countries, especially 
among younger segments of the population, completed 
degrees. Similarly, comparative performance on several 
types of tests and poor results on measures such as the 
National Assessment of Adult Literacy (NAAL) suggested 
that undergraduates in the United States were faring poorly 
on some reasonable standard of literacy compared with 
students in other countries. The Spellings Commission 



report strongly implied that some overall test or tests of 
results should be administered. It specifically mentioned as 
a possible indicator the Collegiate Learning Assessment 
(CLA), a measure of critical thinking, analytic reasoning, 
problem solving, and written communication developed by 
the Council for Aid to Education (CAE). 

More recently, several reports suggest that many college 
students are not learning at the levels expected. One 
particularly strong critique using results from the CLA 
(Academically Adrift: Limited Learning on College Campuses, 
Arum and Roksa) concludes that students are not studying 
very hard and as a result are not learning very much. Arum 
and Roksa, in the finding most commonly noted in the 
popular press, report that more than one third of students 
show no gain in critical thinking over four years in college. 
Unfortunately, attention to this finding has distracted from 
other results of the study and ignored some developments 
in higher education that tell a different and more nuanced 
story that is particularly relevant to small and mid-sized 
private colleges and universities. Specifically, students 
majoring in traditional liberal arts fields demonstrated 
greater gains in learning than others. Also noteworthy is 
the fact that because the study draws from a limited sample 
of 24 institutions, differences in learning across various 
types of institutions could not be fully addressed. Linally, 
and perhaps necessarily, in reporting on their research the 
authors did not take much notice of the wide variety of 
efforts to assess and improve student learning that has been 
taking place in higher education, especially recently. 
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Aurora University (illinois and Wisconsin) is an inclusive 
community dedicated to the transformative power of learning. 
Founded in 1893, Aurora is a private university that enrolls more 
than 4,300 students in its two locations. The university offers 
bachelor’s, master's, and doctoral degrees, www.aurora.edu 

This report provides a picture of how higher education, 
and specifically 47 independent colleges and universities in 
the CIC/CLA Consortium, are working to improve student 
learning by administering and responding to the CLA. 

The Council of Independent Colleges (CIC), an 
association of some 600 small and mid-sized private colleges 
and universities, has been a leader in responding to calls 
for greater efforts in assessing and accounting for student 
learning. Indeed, its efforts to support the development 
of the CLA and encourage its use preceded the Spellings 
report and the controversies surrounding it. In 2002 CIC 
worked with the CAE to identify private institutions that 
would be willing to participate in a beta test of the CLA. In 
2004“2005 a dozen CIC institutions administered the CLA 
and formed the initial CIC/CLA Consortium with support 
from the Teagle Foundation and the Carnegie Corporation 
of New York. Interest in the assessment instrument was 
high enough among CIC members to suggest expanding 
the group for a multi-year period. 

During the period from fall 2005 through spring 2008, 
supported by a new grant from the Teagle Foundation, CIC 
established a Consortium of 33 institutions that agreed 
to administer the CLA. Part of the grant initiative was 
to bring Consortium institutions together annually for a 
summer meeting to exchange experiences of working with 



the CLA and to discuss topics concerning assessment and 
higher education more generally. In 2008 CIC published a 
report on the experience of the Consortium to that point. 
Evidence of Learning: Applying the Collegiate Learning 
Assessment to Improve Teaching and Learning in the Liberal 
Arts Experience. That report was devoted primarily to 
individual descriptions of a dozen or so colleges and 
universities’ experiences with the CLA, a discussion of the 
challenges institutions faced in administering and using 
the CLA, and lessons learned to be shared with other 
institutions. The report was used by many other colleges 
and universities as a set of cautionary tales that informed 
their own assessment efforts. 

From fall 2008 through spring 2011, again with Teagle 
Foundation support, the CIC/CLA Consortium expanded 
to include 47 institutions. (A list of the participating 
colleges and universities and their years of participation in 
the Consortium can he found on page 44.) The summer 
Consortium meetings continued in this phase. In the 
proposal to the Teagle Foundation, CIC President Richard 
Ekman stated, “CIC seeks to firmly embed a ‘culture of 
assessment’ on participating campuses, refine the methods 
used to assess student learning, and establish best practices 
for the improvement of student learning and teaching that 
can be confidently shared with other small and mid-sized 
independent colleges and universities and made known 
to the public.” Also, involvement in this phase required 
colleges to use multiple sources of evidence beyond the 
CLA and participation of faculty members on the CLA 
“team” at Consortium meetings and on campuses. 

True to the spirit of independent higher education, each 
institutional member of the CIC/CLA Consortium was free 
to chart its own course with respect to how to administer 
and use the CLA. There was no uniform expectation about 
the ways in which an institution might decide to change 
its work, if at all, in response to CLA results. At the same 
time, the Consortium offered an annual opportunity 
for representatives from member colleges to meet and 
discuss their progress, and there were ongoing discussions 
and consultation with CAE about the CLA results and 
possible developments in the use of the instrument. With 
few exceptions, the CLA had a considerable impact on 
institutions, resulting in a deeper “culture of assessment” 
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and altered programs and pedagogy in response to CLA 
results. 

The final summer meeting of the TeagleTunded 
Consortium members occurred in August 2010. The 
ending of this phase of CIC’s work with the CLA (and the 
beginning of a new “Pathways” phase discussed in Section 
X) suggested the desirability of some kind of summing up 
of the experiences of the 47 TeagleTunded colleges and 
universities in the Consortium between 2008 and 2011. 
After the August 2010 meeting of the Consortium, colleges 
were asked to submit responses on the following topics 
related to their work with the CLA: 

• Overview: The basic story of what happened on your 
campus, who was involved, what difference using the 
CLA made; 

• Successes: The particular successes and how the day- 
to-day work on your campus has changed as a result of 
working with the CLA; 

• Process: The thorny issues of introducing the CLA on 
campus (particularly to and by faculty), administering 
the test, and reporting and discussing CLA results; 

• Challenges: The difficulties and even disappointments 
experienced, since these are sometimes as important 
to understand as successes; 

• Consortium: The impact of participating in the 
Consortium (did it make a difference?); and 

• Looking ahead: Plans for using the CLA beyond this 
academic year. 

A select group of presidents of Consortium institutions also 
was asked to comment on the changes resulting from the 
initiatives and their hopes and plans for future development 
of assessment efforts. 

Summarizing and doing justice to the work of each 
of the institutions would require a much larger volume 
(or volumes). As one might expect, there is a variety of 
distinctive stories about institutional experiences and 
results. Instead of focusing on each institution separately, 
what follows is a description of some of the major trends 
and issues among the colleges, responses to results and 
problems, successes and challenges, and remaining 
questions and issues. In each section, the examples given 
from specific institutions almost certainly have parallels 
at other colleges as well, even as institutions responded in 




Averett University (Virginia) is a private, co-educational 
institution affiliated with the Baptist General Association of 
Virginia. Averett offers undergraduate curricula blending the 
liberal arts and sciences with professional preparation, as well 
as graduate programs in education and business. Averett is a 
community that values cultural, individual, and racial diversity. 
www.averett.e6u 



different ways to their own circumstances. 

The next section of this report offers a brief overview 
of the CLA. The sections that follow discuss some of 
the ongoing challenges of assessment generally and of 
administering and using the CLA in particular, the direct 
and indirect impact of the CLA on different institutions, 
including the increasing interest in the “CLA in the 
Classroom” program, and what the future might look like 
for CIC member colleges — including CIC’s next CLA- 
related initiative, the Pathways Project. The concluding 
section summarizes the general results and lessons learned 
from the CIC/CLA Consortium. 

The various sections do not provide sharp distinctions 
between and among institutions nor do they reflect some 
strict categorization of what goes on within any college 
or university. Rather, the sections point to the various 
aspects of the introduction and development of most 
new campus initiatives. For example, the distinction 
between “direct” and “indirect” impact is not always 
clear, and at almost every institution both can be found. 
The introduction of a new initiative, in this case using 
the CLA to assess student learning, sometimes produces 
immediate actions, or sometimes it marks the beginning 
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Barton College (North Carolina), founded in 1902, is a 
four-year, private iiberai arts coiiege that offers a wide range 
of professionai and iiberai arts programs ieadingto the 
baccaiaureate degree, as weii as the MEd in eiementary 
education. The coiiege serves approximateiy 1,150 students 
and boasts a soiid internationai travei study program and an 
active honors program, www.barton.edu 

of a process of considering a broader range of issues and 
how they might be addressed — or both. A new initiative 
typically precipitates discussion, and sometimes conflict, 
about institutional purposes and means for evaluating 
change. These discussions occasionally or even often have 
long-ranging influence in that they provide new ways 
of thinking about and understanding the work of the 
institution. 

And of course, new initiatives almost never move 
from proposal to action to results without encountering 
challenges along the way. Introducing the CLA 
necessitated finding ways to overcome practical issues of 
administration and institutional process, as well as the very 
real questions of how to interpret and what to do with any 
given set of results. Over several years, not surprisingly, 
many institutions developed ways of more effectively 
administering the CLA and using it for programmatic and 
pedagogical change. The annual meetings of members 
of the Consortium were almost universally cited as 



being a useful source of tips and tactics for making the 
administration and use of the CLA easier and more 
productive. 

As will be seen from the descriptions in the succeeding 
sections, in almost every case the CLA had both direct and 
indirect impact on the programs, pedagogy, and thinking 
at these institutions. On many campuses workshops on the 
pedagogy of writing and critical thinking were developed 
in response to CLA results. Programs and pedagogy 
about writing and critical thinking, among other things, 
were reexamined and changed. Faculty members became 
interested in using the CLAs notion of an “ill-structured” 
problem as an organizing concept for understanding 
critical thinking, and many specifically wanted to create 
performance tasks for their courses (following the “CLA 
in the Classroom” approach). The consideration of these 
initiatives very often involved broader discussions of 
institutional purpose, the aims and value of assessment, 
and best ways to promote and evaluate change. These 
broader discussions often produced cultural changes — 
changes in perspective that accompanied programmatic 
and pedagogical shifts. 

There is no simple blueprint, roadmap, or formula 
for establishing a successful assessment program, with or 
without the CLA. This endeavor provided the opportunity 
for making improvements, especially as institutions drew 
from the experience of colleagues through the CIC/ 

CLA Consortium. There is greater understanding and 
acceptance of the CLA and assessment generally among 
these colleges, as well as many programmatic changes and 
improvements. Certainly many questions remain, but these 
are in sharper focus because of the introduction of the 
CLA in these small and mid-sized independent colleges 
and universities. The willingness of these institutions to 
experiment, interact, and learn from their experience 
in the CIC/CLA Consortium points to a continuing 
leadership role for CIC in the national discussion on 
improving student learning. ■ 
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II. CRITICAL THINKING AND PROBLEM SOLVING IN AN IMPERFECT 
WORLD: THE COLLEGIATE LEARNING ASSESSMENT 



“The CLA is a unique instrument and, as such, can be an important 
piece, albeit not the only piece, of an effective assessment plan.” 

-Cynthia Zane, President, Hiibert Coiiege 



T he CLA was developed by the CAE as a way of 
measuring institutional contributions to gains in 
student learning. The CLA is one of the first widely-used 
testing instruments to measure student learning directly by 
capturing the contribution of the institution to students’ 
cognitive development. The test provides a standardized 
measure of “higher order” cognitive skills and competencies 
that are often aligned with the general education goals of 
the undergraduate curriculum: critical thinking, analytic 
reasoning, problem solving, and written communication. 

The CLA instrument asks first-year and senior students 
to evaluate arguments and to write a response to practical 
problems using provided materials that do not yield any 
precise or correct answer. Specifically, the CLA uses three 
key measures to assess student abilities: 

• Make an Argument: The ability to take and justify a 
position on an issue. 

• Critique an Argument: The ability to evaluate an 
argument for how well-reasoned it is. 

• Performance Task: The longest section of the test 
asks the student to complete a real-world task, such 
as preparing a briefing report, using a set of provided 
materials. 

Thus, in making or critiquing an argument the test 
might state, “In our time, specialists of all kinds are highly 
overrated. We need more generalists — people who can 
provide broad perspectives,” and ask the test-taker, in 
45 minutes, to agree or disagree with the statement and 
explain the reasons for his or her position. 

The performance task, the best-known and most 
well-regarded element of the CLA, assigns the student 
a role in a scenario and asks him or her to write on the 



basis of provided materials an analysis of the problem and 
response to it. For example, a test taker might be asked 
as a consultant to a mayor to respond to criticism by an 
opponent in an election. The opposing candidate has 
criticized the mayor’s proposal to increase the number 
of police officers in order to reduce crime. In suggesting 
how the mayor should respond the test taker is provided 
with several different kinds of documents, police tables, 
crime statistics, one of the opponent’s exhibits, etc. Taken 
together, the documents do not (obviously in the case of 
the opponent’s exhibits) support the mayor’s claim or for 
that matter the claim of the opponent. This kind of 




Bethel University (Minnesota) is a leader in Christ-centered 
higher education with approximately 6,400 students from 47 
states and 32 countries enrolled in undergraduate, graduate, 
seminary, and adult education programs. Based in St. Paul, 
with additional seminary locations on both coasts. Bethel offers 
rigorous bachelor’s and advanced degrees in nearly 100 fields. 
www.bethel.edu 
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Cabrini College (Pennsylvania) is a residential college founded 
in 1957 by the Missionary Sisters of the Sacred Heart of 
Jesus. Cabrini offers 1,400 undergraduate students more 
than 30 majors, pre-professional programs, concentrations, 
and minors. More than 2,000 students are in graduate and 
professional studies programs at the main campus and at 15 
off-site locations, www.cabrini.edu 

“ill-structured” problem is the hallmark of the performance 
task. 

While certainly not the only means of assessing student 
learning, the CLA provides an assessment instrument 
that has considerable intuitive plausibility. Critical 
thinking in terms of analyzing arguments or constructing 
them in general or in the face of an ill-structured 
problem — and writing well about them — are widely held 
aims for undergraduates. As John Reynders, president of 
Morningside College puts it, “the CLA has captured the 
interest of the faculty. This has led to discussions on how 
better to deliver two of our student learning outcomes, 
writing and critical thinking.” Moreover, unlike most tests 
of student learning that use multiple choice, true-false, or 
short-answer questions, the CLA uses written, student- 
constructed responses to open-ended assignments. In 
contrast to subject domain instruments that test students’ 
knowledge of particular disciplinary content, the CLA 
poses problems that students must address by constructing 
their own arguments for or against a particular position in 
general or, in the performance task, evaluating evidence, 
synthesizing information, and drawing conclusions about 
a real-world problem. Research scientist Marc Chun of 



Carlow University (Pennsylvania), the first Catholic, women- 
centered, liberal arts university in Pennsylvania, currently 
enrolls 2,200 students, has a student-faculty ratio of 10-1, 
and offers small classes and individual instruction. The 
student population is culturally diverse and contains traditional 
and nontraditional students, and more than half are first 
generation, www.carlow.edu 

the CAE said, “Our goal has been to support schools in 
aligning the teaching, learning, and assessment of critical 
thinking and other higher order thinking skills. And what 
has been really exciting is to see how quickly this happened 
at some institutions.” 

The scores on the various elements — for making and 
critiquing of arguments, the performance task, and the 
quality of writing — are assigned based on scoring rubrics 
or criteria that provide a standardized basis for measuring 
test results. These scores may then be analyzed in two ways. 
First, the scores on these various parts of the test and the 
score for the quality of writing can be analyzed in terms of 
expected performance based on students’ entering scores 
as first-year students on other standardized tests (SAT or 
ACT). This adjusted score shows how students performed 
relative to their ability — at, above, or below expected — for 
samples of first-year and senior students. 

Second, the test results produce a second kind of 
score, an institutional “value-added” score. The CLA is 
administered to cross-sectional samples of students in their 
first and senior years. Considering the scores of seniors 
and first-year students together and using student SAT 
(or ACT) scores as a control, the CLA reports whether 
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Charleston Southern University (South Carolina), founded in 
1964, is one of South Carolina’s largest accredited, independent 
universities, enrolling approximately 3,200 students. Affiliated 
with the South Carolina Baptist Convention, the university’s 
vision is to be nationally recognized for integrating faith in 
learning, leading, and serving, www.csuniv.edu 

students, when measured as a group, perform at, above, 
or below expected levels. This gives an indication of 
whether the institution performed at/above/below expected 
performance and the “value-added” to students’ skills 
between the first and fourth year. The value-added score 
indicates how the aggregated first-year to senior gains on 
these higher order skills compare with other institutions. 

Thus, the CLA provides one of the first “value-added” 
measures that compares what students can do when 
they start college with what they can do when they 
finish (controlling for initial ability) and thus arguably 
demonstrates institutional contributions to student 
learning. It provides a reliable institutional value-added 
score that can be tracked over time and benchmarked 
against similar institutions. By making the institution 
rather than an individual student its primary unit of 



analysis, the CLA keeps the focus on how the institution as 
a whole contributes to student cognitive development. 

It is important to note that the CLA is intended as 
one source of evidence in assessment of student learning, 
not as the sole measure. CLA results can and should be 
combined with other institutional data to determine 
factors that promote student learning and growth. Cynthia 
Zane, president of Hilbert College, notes, “The CLA is a 
unique instrument and, as such, can be an important piece, 
albeit not the only piece, of an effective assessment plan.” 
For example, an institution might compare scores on the 
CLA, an outcomes measure, with reported experiences 
on the National Survey of Student Engagement, a process 
measure. As noted previously, since the summer 2008 
meeting of the CIC/CLA Consortium, participating 
colleges and universities have been expected to triangulate 
data from several sources in assessing student outcomes. 

This kind of triangulation is extremely important. It is 
simply good practice to gather data from several sources, 
using different methods, to obtain a valid picture of what 
is going on. Moreover, as will be discussed in more detail 
later, often faculty members and administrators wondered 
how CLA results could be translated into proposals for 
change or strengthening of existing practices. This is 
especially true for faculty members who may see little or no 
connection between their work in a single class or even a 
major and the larger pattern of CLA results. It is therefore 
important that CLA scores be combined with other 
measures to round out the picture of what is happening 
across the campus with regard to student learning. The 
good news in the experience of the Consortium is that 
all institutions, to a greater or lesser degree, have sought 
to do some kind of triangulation and are likely to do so 
in the future. The CLA has served as the anchor in the 
assessment portfolio of these institutions. ■ 
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III. THE CLA IN CONTEXT: ASSESSMENT BEYOND GRADES 



“Building a culture of assessment takes time and is most effective when 
grassroots efforts combine with administrative decision and support.” 

-Frederik Ohies, President, Nebraska Wesleyan University 



s important as putting the CLA in the context 
of other measures of student learning is, it is also 
important to see the CLA in the context of the everyday 
practices of institutions of higher education. Despite several 
decades of discussion concerning the assessment of student 
learning, conducting systematic assessment — beyond the 
traditional grading of pieces of student work and overall 
course performance by faculty members — does not come 
naturally to colleges and universities. It is still common 
for faculty members to say that they are already doing 
assessment by grading. How well a student has performed 
is traditionally assessed through the grade for a course and 







College of Saint Benedict (Minnesota) for women and Saint 
John's University (Minnesota) for men are liberal arts colleges 
ranked as two of the top three Catholic colleges in the nation. 
They share one academic program, and students attend classes 
together on both campuses. The combined enrollment of more 
than 3,800 students makes CSB and SJU one of the largest of 
the nation’s liberal arts colleges, www.csbsju.edu 



the compilation of grades and an average on a transcript. 

It is largely assumed that in taking the courses that fulfill 
general education requirements and completing a major a 
student is making gains in knowledge, skills, and abilities, 
thereby achieving (to a greater or lesser degree) the 
outcomes sought by the institution. Therefore, demands 
for more or other assessments of student learning beyond 
grades is often not an idea that is seen by members of the 
faculty as necessary. 

Two other features of the typical institutional context 
of colleges or universities are also relevant to the way 
in which assessment, through the CLA or any other 
measure, might be received. First, many faculty members 
think of their teaching role in the context of a discipline. 
They teach what they are expert in — biology, history, 
economics, literature, etc. They are not trying to teach 
students directly to “think critically,” but rather are trying 
to teach students to think critically about a given subject 
matter. The extent that learning rigorous analytic skills 
in a field can be applied to all learning is the goal of a 
liberal arts education. But too often the emphasis falls 
on subject matter to the exclusion of transferable critical 
thinking skills. Moreover, even when faculty members are 
sympathetic to the kind of measurement introduced by the 
CLA, they also legitimately may wonder what its results 
imply for their individual teaching and the work of their 
departments. 

Second, it is important to keep in mind that colleges 
and universities work through a system of shared 
governance. A president or dean typically does not and 
cannot dictate how individual departments and faculty 
members will participate in or respond to assessment 
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College of St. Scholastica (Minnesota) is a Catholic 
Benedictine institution founded in 1912 that enrolls more than 
3,300 students. The college awards bachelor’s and master's 
degrees, and two doctoral degrees, www.css.edu 




Dominican University (Illinois), founded in 1901, is a 
comprehensive, coeducational Catholic institution offering 
bachelor's degrees through the Rosary College of Arts and 
Sciences and master's degrees through the Graduate School 
of Library and Information Science, the Brennan School of 
Business, the School of Education, and the Graduate School 
of Social Work. The university also offers a doctoral degree in 
library and information science, www.dom.edu 

initiatives or results. In assessment, as in many aspects of 
college and university policy and practice, consultation 
and consensus building are the rule. The presumption 
that governs questions of individual faculty members’ or 
departments’ efforts in teaching is the classic principle 



of “Lehrfreiheit,” or “freedom to teach” — the autonomy of 
faculty members and their departments in deciding what to 
teach (and how). This suggests that ultimately if the CLA 
or any assessment regime is to have the desired effect — the 
consideration of evidence and action for improvement 
based on it — faculty involvement in the discussion of 
the regime is necessary and desirable. The requirement 
of faculty participation in the teams on CIC/CLA 
Consortium member campuses recognized the importance 
of faculty involvement in the process. 

These features of colleges and universities and the 
institutional context into which the CLA has been 
introduced complicate what is always the potentially 
difficult challenge of analyzing data, interpreting results, 
and deciding what to do in response. Conducting 
assessment and gathering data, in any endeavor and 
especially in higher education, rarely lead obviously 
and directly to action. Data need to be interpreted; 
conversations need to take place; and decisions need to 
be made about what, if anything, can or should be done 
in response to the findings. The arrival of some data or a 
report inevitably leads to the question, “Now what?” 

In the case of the CLA, these “Now what?” 
conversations between and among faculty members 
and administrators naturally began with the data. The 
questions of whether the measure or the results are valid 
or reliable were often raised immediately. Even before 
questions such as these were raised at the institutions in 
the Consortium, it was typically necessary to discuss what 
the CLA is and what it purports to show. Further questions 
often arose about the student sampling methodology and 
how the data were gathered, a particular issue with the 
CLA. Finally, it was often the case that a discussion of 
the instrument, the data, or the data-gathering process 
would quickly expand to a more general discussion about 
assessment, its purpose and validity, and how it might best 
be done. 

On any campus a conversation about assessment results 
evolves into the question of what, if anything, needs to be 
done in some shorter or longer term. What do the data tell 
us and how should we respond? Even assuming the shared 
desire to respond to results, it might be unclear what an 
appropriate response might be. Positive or negative results 
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might, respectively, be seen as confirming institutional 
practices or signaling the need for some immediate action. 
Alternatively, some specific response might seem apparent 
to administrators and faculty members. Moreover, the 
intersecting paths of conversation and action are hardly 
predictable. Institutions may start taking one direction 
in conversation or action and end up discussing or doing 
something entirely different. 

The reports of the CIC/CLA Consortium institutions 
illustrate a wide range of responses to the question, “Now 
what?” In almost every case the CLA had an impact on 
these colleges and universities, although these impacts 
varied in substantial ways. What does emerge overall is the 
seriousness of the attempts to understand what students 
were achieving at these institutions and to wrestle with the 
questions raised by the CLA. 

Later sections of this report describe some of the ways 
in which the institutions reacted to the CLA results, 
with examples of each. At the same time, however, even 
where there were demonstrable successes, in many cases 
the administration and use of the CLA faced significant 
challenges. These included practical problems in 
administering the test, political problems in persuading 



faculty members to become involved in the initiative 
directly or indirectly, and persistent questions about the use 
and usefulness of the CLA. ■ 




Drake University’s (Iowa) mission is to provide an exceptional 
learning environment that prepares students for meaningful 
personal lives, professional accomplishments, and responsible 
global citizenship. The Drake experience is distinguished by 
collaborative learning among students, faculty, and staff 
and by the integration of the liberal arts and sciences v/ith 
professional preparation, www.drake.edu 
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IV. CHALLENGES: THE LOGISTICS, POLITICS, AND PUZZLES 
OF THE CLA 



“Institutions need to find a way to embed assessment measures like the 
CLA into capstone courses for seniors. When students can connect these 
metrics to performance in the majors they tend to be more motivated 
to give their best effort. This allows for a more accurate snapshot of 
institutional and program effectiveness.” 

-Michael Bassis, President, Westminster College (Utah) 



A nyone familiar with the process of reform in 

colleges and universities knows that change is often 
slow, uneven, and occasionally even futile. Introducing 
the CLA to the CIC/CLA Consortium institutions 
has been no different. What follows is a description of 
some of the challenges and obstacles these institutions 
faced and how some dealt with them. These examples 
are illustrative of what many CIC/CLA Consortium 
institutions encountered, and almost all reported some mix 
of success and challenge. For the most part, experience in 
administering the instrument and continuing conversations 
helped institutions meet the challenges and overcome the 
obstacles. 

Perhaps the most significant challenges confronted 
by the CIC/CLA Consortium institutions were getting 
students to take the test and to take it seriously. The cross- 
sectional method for administering the CLA requires that 
random samples of 100 first-year and 100 senior students 
take the test in order to produce some measure of “value- 
added” on the components and overall results for gains 
in critical thinking. This means that students must be 
recruited to take the test and their performance needs to 
reflect their sincere efforts. 

Recruiting an adequate sample produced a set of 
significant challenges for many campuses. There were 
basic logistical issues in finding times and available spaces 
and computers for students to take the test — claims on 
time, space, and coordination that can be difficult on a 



small campus. The College of Saint Benedict and Saint 
John’s University reported that administering the CLA 
was a challenge due to “student schedules, extracurricular 
activities, and availability of testing labs on campus.” 
Similarly, Drake University reported logistical challenges 
in finding computer labs that were not already booked for 
classes, as well as “difficulties in recruiting enough students 
to ensure an adequate sample size that is likely to garner 
representative results.” In these and in other cases, careful 
attention to details and embedding the test into other 
formally scheduled classes or activities (as described later) 
helped resolve these problems. 

Experience showed that occasionally even a slight 
adjustment in test administration can produce changed 
(and more, or less, valid and understandable) results. At 
Jamestown College, it was relatively easy to recruit seniors 
to take the test on an annual “assessment day” in the 
spring. However, the first year the CLA was administered, 
students in some of the larger majors (such as nursing) 
could not take the test due to other commitments. The 
results of this administration were inconsistent with other 
assessment data — and in the “wrong” direction, seeming 
to show less positive results for students. The second year, 
Jamestown “sampled seniors by major so we would have 

a better cross-section of students Our results were 

significantly better and more consistent with what should 
be expected between first-year and senior students.” 
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Franklin Pierce University (New Hampshire) prepares its 
students to become active, engaged citizens and ieaders 
of conscience. Whether at the undergraduate, graduate, 
or professionai ievel, its academic programs are grounded 
in a iiberai arts tradition and are unified by the theme of 
the individuai and community. The university enroiis 1,500 
undergraduates and 1,500 graduate students. 

WWW. franklinpierce. edu 

If logistical problems regarding scheduling, computer 
and room availability, and sampling are worked out, there 
is then the challenge of getting students to show up to 
take the test. There frequently needs to be some incentive 
to encourage students to report to the test site if, as was 
mostly the case, participation is voluntary. On the other 
hand, many of the Consortium institutions embedded 
the test in existing settings, making the test in some 
cases mandatory for students. For example, Westminster 
College (Missouri), administered the test “to all beginning 
freshmen during their first week on campus in the fall.” 
The CLA is administered to seniors “as part of the spring 
Assessment Day that is scheduled each year in March.” 
Other institutions embedded the CLA into existing 
courses. Bethel University (Minnesota) administered the 
test in some of its freshman-level courses and in senior- 
level general education courses. Since many CIC/CLA 
Consortium institutions have required first-year seminars 
and capstone courses, they have natural settings in which 
to administer the test. Perhaps the most emphatic form 
of requiring students to take the test comes from the 
University of Charleston (West Virginia): “It became 
a graduation requirement that students participate in 



the University’s assessment activities.” This requirement 
was supplemented by embedding testing into course 
and orientation settings, and having the “institution’s 
commitment to assessment. . .reflected on students’ program 
evaluation or degree audits.” 

Where requiring the test is not, for whatever reason, 
possible or desirable, institutions have used incentives or 
persuasion or both. Initially, College of Saint Benedict 
and Saint John’s University “randomly selected student 
names and invited them to take the CLA. The problem 
has been that too few students who were assigned to take 
the test actually came and took the test.” The institutions 
responded to this problem and increased participation 
by a mix of incentives and persuasion. They gave a small 
stipend (a book store certificate) to students. This incentive 
was supplemented by the creation of a group of six student 
workers to recruit test takers and having faculty members 
promote the test in courses, particularly in the first year. 
The numbers of participants rose dramatically as a result 
of these efforts. Giving specific compensation as an 
incentive is a tactic used by a number of colleges, including 
offering lottery tickets for a computer or other prize(s). The 
exchange of pizza or other popular food for participation 
was used by several institutions. One interesting initiative 
at the University of Evansville (Indiana) was to give the 




Hastings College (Nebraska), founded in 1882, is a private, 
four-year liberal arts institution affiliated with the Presbyterian 
Church (USA). A total of 64 majors in 32 areas of study and 
12 pre-professional programs are offered to more than 1,190 
students, www.hastings.edu 
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Hilbert College (New York) is a private four-year college 
founded In 1957 In the Franciscan tradition. With nearly 1,100 
students, Hilbert offers career-focused majors. Including one of 
the top criminal justice programs In the region, and more than 
50 minors and concentrations, www.hilbert.edu 

top three scorers on the CLA, administered in the required 
first-year seminar, vouchers to spend at the campus store. 

Finally, assuming successful logistics and the 
participation of students in the CLA, if the results are 
to be meaningful it is desirable that students put forth 
sincere effort in taking the test. Since there is no grade 
on the transcript or other incentive — unlike the SAT or 
GRE, the CLA is a “low stakes” test — students may not be 
motivated to take the CLA seriously. This was particularly 
true of seniors, as Southwestern University (Texas) noted 
in an otherwise very positive report on working with the 
CLA, “The main difficulty. . .is recruiting seniors to take 
the instrument and, once recruited, to take the assessment 
seriously. We have tried various methods for increasing 
participation such as financial incentives, but we continue 
to struggle to get seniors to sign up and participate.” 

While institutions made it clear that the test was not a 
measurement of the performance of individual faculty 
members or students, students appeared often to want the 
CLA to be a test that measured their individual progress 
over four years — some even wanted to be able to use the 
test results in their portfolio to prove how much they had 
learned. 

Many colleges reported that they were not confident 
that students, especially seniors, were trying as hard as 



they could to do well. At Marian University (Wisconsin) 
initial difficulties in recruiting and motivating seniors 
reduced the sample size and actually produced results that 
showed no, or even negative, value-added. Faculty members 
and administrators made “a more concerted effort. . .to 
communicate with the seniors regarding the importance 
of their participation with the instrument. Advisors have 
also helped in recruiting seniors to take the test.” Similarly, 
at Barton College (North Carolina) the initial assumption 
that “all we had to do was describe the CLA (giving 
samples of the writing prompts) to the faculty members 
and the chairs and deans, in order to get the students to 
take the assessment seriously” quickly gave way to “the 
chief academic officer visiting every section of the first-year 
seminar” with “a direct appeal to the students to take the 
CLA seriously, explaining why it was important to the 
college and to them. From then on, the level of engagement 
with the process improved enormously.” Michael Bassis, 
president of Westminster College (Utah), suggests that 
institutions “find a way to embed assessment measures like 
the CLA into capstone courses for seniors. When students 
can connect these metrics to performance in the majors 
they tend to be more motivated to give their best effort. 




Illinois College Is a private. Phi Beta Kappa, liberal arts 
college. With an enrollment of approximately 1,000 students, 
Illinois College offers a residential liberal arts curriculum 
of more than 45 majors in the humanities, social sciences, 
and natural sciences. A rich array of experiential learning 
opportunities including study abroad, student-faculty research, 
internships, community service projects, and Division III 
athletics complement the academic program, www.ic.edu 
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Indiana Wesleyan University is an evangelical Christian 
comprehensive university founded in 1920 and committed to 
liberal arts and professional education. Almost 3,300 students 
are enrolled in traditional on-campus programs and about 
12,500 adult learners attend classes at education centers in 
Indiana, Kentucky, and Ohio and online, www.indwes.edu 

This allows for a more accurate snapshot of institutional 
and program effectiveness.” 

Students were not the only ones who needed to be 
persuaded to become involved in the CLA initiative. 
Faculty members, not surprisingly, raised questions about 
the validity of the CLA and standardized assessment 
more generally. For example, at Drake University, faculty 
members doubted “the validity of measuring something 
as fluid as critical thinking skills via a standardized 
instrument.” These doubts were not lessened by the 
occasional instances of wide swings in year-to-year scores. 
For example, at Texas Lutheran University the “large year- 
to-year variation in value-added scores,” from being above 
the 80 th percentile to below the 40 th, “make it difficult 
for many at TLU to place high confidence in CLA scores.” 
At the same time these swings may easily be a function of 
issues of administration, sampling, and motivation. 

Although faculty “buy in” concerning the CLA 
continues to be a challenge on some campuses, ongoing 
experience with the CLA has in most cases led to 
increasing acceptance and even enthusiasm about its use 
as part of an institution’s assessment program. Several 
institutions reported that faculty members’ support grew 
the more they talked about the CLA and as some became 



involved in its administration. For example, at Westminster 
College (Missouri), “proctors are chosen from the broader 
faculty. As a result, there is a sense that faculty members 
generally appreciate the assessment activity better after 
looking over the shoulders of students as they complete 
the CLA each term.” Leaders at Allegheny College 
(Pennsylvania) report, “It is also fair to say that our CLA 
testing has helped cultivate more of a culture of assessment 

at the college We expect the CLA to be a very useful 

barometer by which to read the success of our curricular 
change.” 

Even as most institutions have dealt successfully with 
difficulties of administering and using the CLA, it is not 
surprising that at a very few institutions the combination 
of difficulties administering the test, working with students 
and faculty members, and interpreting results led to 
consideration of whether to continue use of the CLA. This 
sentiment was expressed at Illinois College: “Given the 
cost of the CLA, administrators are showing reluctance to 
continue its use if results cannot be explained clearly and 
reliably for a variety of audiences.” Finally, Juniata College 
(Pennsylvania) has decided to drop the CLA altogether as a 
result of “difficulties in recruiting students and finding ways 
to utilize the results in a meaningful way,” issues of “buy-in 
from faculty and staff,” and a lack of correlation between 
CLA and other assessment results. 

No assessment instrument is perfect, and its limitations 
are compounded by the kinds of logistical and political 
challenges described above. Indeed, it would be unusual 
if the introduction of a new assessment instrument went 
smoothly, was unquestioned, and the data immediately 
could be translated into clear, positive actions. The 
responses to challenges arising in the administration and 
use of the CLA indicate that ongoing conversation and 
more experience are the best means for meeting these 
challenges. One particularly good vehicle for having and 
broadening these conversations and sharing experiences, 
discussed in Section IX, has been the CIC/CLA 
Consortium’s annual meetings and activities. Despite the 
challenges and obstacles for almost all of the Consortium 
institutions, the introduction of the CLA led to significant 
direct and indirect impacts and positive institutional 
change. ■ 
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V. A SENSE OF URGENCY: CHANGING THE TEACHING OF WRITING 
AND CRITICAL THINKING 



“The CLA has led to development of a faculty culture of understanding 
that their responsibilities go beyond ‘coverage of disciplinary 
knowledge’ to include the essential development of critical thinking 
and communication.... Even more telling is that faculty conversations 



surrounding curricular and outcome 
vocabulary and concepts.” 



F or a number of colleges and universities in the CIC/ 
CLA Consortium, the administration and results of 
the CLA suggested the desirability of taking immediate 
action. The answer to “Now what?” was to do something 
to change and improve a specific part of the curriculum. 

Not surprisingly, because the scores on the CLA specifically 
“grade” writing and critical thinking, these were the areas 
that many Consortium institutions sought to address. 
Changes in programs and ways of teaching writing, a 
sharper focus on what critical thinking meant, and how 
it could be better promoted in courses and programs were 
often immediate responses to CLA results. 

At Ursuline College (Ohio) disappointing CLA results 
reinforced an existing “general dissatisfaction with student 
writing.” This in turn led to the creation and distribution 
of a “glossary of common terms that includes ‘make an 
argument,’ ‘critique an argument,’ distinguishing between 
‘writing on demand,’ and formative and summative 
writing.” What is especially noteworthy about the idea of 
a glossary is that it creates the potential for more common 
understandings among faculty members across departments 
and programs. The development of the glossary was part 
of a more general reorganization of the writing center and 
its program on campus. More support for teaching writing 
was put into place, specifically the creation of a liaison from 
the English department to assist in “faculty development 



measures are utilizing CLA 

-Norval Kneten, President, Barton College 



workshops providing individual support to instructors for 
developing and assessing writing assignments.” Faculty 
members now feel they have a much better understanding 
of teaching and assessing writing. 

Barton College (North Carolina) followed a similar 
path and developed its own campus version of the CLA, 
which led to some dramatic changes. “After reading 
samples of the actual writing of Barton seniors. . .and 
after extensive discussions involving faculty members, 
staff, and trustees, the whole community decided that 
improving student writing should be the major curricular 
focus.” The faculty members at Barton created a standard 
rubric for evaluating writing and the development of 
electronic student portfolios that allow a campus-wide 
assessment of writing. The results have been similar to 
those of Ursuline’s. “Using the standard rubric rating scale 
and electronic portfolios allows us to analyze and use the 
large amount of data effectively for program assessment 
and planning.” These actions in response to the CLA 
have also had the effect of building broader community 
understandings, even beyond the faculty. “[W]e have found 
that talking about the CLA as part of our institutional 
focus on developing communication and critical thinking 
skills resonates well with prospective students and parents, 
board members, and prospective donors.” 
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Jamestown College (North Dakota), established in 1883, is a 
private, liberal arts college granting bachelor of arts, bachelor 
of science, and bachelor of science in nursing degrees as well 
as master’s degrees in education. Jamestown College offers 
more than 40 areas of study. It features a four-year graduation 
guarantee and a guaranteed internship, www.jc.edu 

In a number of cases, the CLA results led to more 
targeted curricular changes related to writing. Seton Hill 
University (Pennsylvania) reported that because CLA 
results suggested that “students may be struggling with 
writing skills related to critical thinking, the university 
began an intensive writing program that provides an 
additional intensive writing class within each major.” This 
involved a substantial redirection of curricular focus and 
resources. These changes have led to some improvement, 
in no small measure because of greater discussion 
and attention to writing in relationship to the CLA 
results. A similar change occurred at Stonehill College 
(Massachusetts), where the “CLA has been particularly 
helpful in providing data that led to proposed changes in 
the general education curriculum — particularly around 
the development of writing intensive courses, both in the 
first year and within each major.” These changes were 
not only a response to CLA results but also to data from 
the National Survey of Student Engagement (NSSE) that 
suggested a need to “increase academic challenge and 
rigor,” and a stronger emphasis on writing was seen as a 
means to accomplish this goal. 

An even more focused change occurred at William 
Woods University (Missouri). Although faculty members 



were skeptical of the CLA as a measure of critical thinking 
(“How can a test measure critical thinking and analytic 
reasoning?”), several members of the English faculty 
“recognized the implications of the test” for writing 
and “realized they would need to change the freshman 
composition curriculum to meet the writing standards 
of the CLA.” They also went further and “surveyed all 
faculty on the types of writing assignments they required.” 
The result was a shift in composition courses to making 
and critiquing an argument in writing assignments and 
broader faculty development workshops aimed at creating 
“similar expectations for the quality of writing.” Although 
the efforts have yet to result in dramatic improvement in 
CLA scores, “faculty stated they believed writing improved 
between 2007 and 2010.” 

Einally, with respect to writing, occasionally an 
institution would respond directly to the CLA results 
but conceptualize the issue in a distinctive way. The 
programmatic focus at Ursinus College (Pennsylvania) in 
the teaching of writing, for example, shifted after looking 
at first-year results. Eaculty members concluded that many 
of the apparent writing deficiencies were accompanied, 
and perhaps caused by, deficiencies in reading. In its first- 
year seminar faculty members are now “in the process of 
initiating a review of how we can improve reading skill” in 
seminars for first-year students. This was also accompanied 
by a broader evaluation of writing in capstone courses, 
again in response to CLA results, “as part of our assessment 
approach to address problems with ‘quick fixes’ that lead to 
larger changes. CLA has helped us address issues without 
waiting for lengthy reports.” 

Three of the four scores on the CLA are related to 
critical thinking, so it is no surprise that a number of 
colleges used the results to reexamine and change their 
approach to teaching thinking skills. Institutions made 
both curricular and pedagogical changes directed at 
improving critical thinking (and CLA scores). Eor example, 
disappointing CLA scores were the motivation for Cabrini 
College (Pennsylvania) to “increase faculty development 
around critical thinking” and create an entirely new 
general education curriculum that “centers around aspects 
of critical thinking in an engaged world.” The new 
curriculum was piloted in 2008-2009 and implemented the 
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John Carroll University (Ohio) is a iiberal arts university 
grounded in the Jesuit, Catholic tradition. The university has 
some 3,100 undergraduates and 700 graduate students. 

Originally founded as St. Ignatius College in 1886, the 
university was renamed in 1923 to honor America’s first 
Catholic bishop, John Carroll of Maryland, www.jcu.edu 

following year. The understanding is that “the CLA is an 
essential piece in the assessment of the new curriculum,” 
and will indicate how much effect these curricular changes 
might be having. 

Similarly, Nebraska Wesleyan University has made 
critical thinking its first priority in a several-year process 
of strategic planning and curricular reform. Led by a CIC/ 
CLA workgroup composed of individuals in the areas 
targeted by the institution’s Consortium goals, a survey of 
faculty definitions of critical thinking resulted in faculty- 
wide workshops on the teaching of critical thinking. These 
discussions in turn became a key part of a three-year 
curricular initiative, “Success for Every Student,” that will 
include the embedding of CLA testing in first-year and 
senior seminars. The university will then analyze data from 
the CLA and other sources to estimate the effects of its 
curricular changes. 

For other CIC/CLA Consortium institutions the CLA 
results led to a focus on pedagogy, sometimes addressing a 
more specific aspect of the CLA assessment. For example, 
while Hilbert College (New York) faculty members were 
pleased with CLA results, they initiated discussions at 
various levels about mission and pedagogy. Specifically, 
faculty members “saw that students were relatively weaker 



in developing skills to create their own arguments. They 
viewed the ‘make-an-argument’ writing task from the 
CLA. . .as an important indication of creative as well as 
critical thinking.” The faculty responded by reexamining 
“course syllabi to see how current assignments may relate 
to these skills and where assignments might be revised 
or added to bolster argument construction.” The broader 
effect of these efforts has been greater understanding of 
how assessment results can both reinforce the positive work 
faculty do as well as point to areas that can be improved. 

At Loyola University New Orleans (Louisiana), on the 
other hand, students “did well when making an argument 
but performed less well when asked to critique an argument 
that was presented in the test. This result got the attention 
of the faculty as an indicator of student weakness that 
needed attention.” At the time of initiating the CLA, 
Loyola had reformed its core curriculum, including the 
implementation of first-year seminars as part of a broader 
first-year experience initiative. The changes provided a 
good location for attention to critical thinking in terms 
of critiquing an argument, and it produced results: “Since 




Juniata College (Pennsylvania) is an independent, 
coeducational college of liberal arts and sciences founded in 
1876 by members of the Church of the Brethren to prepare 
individuals ‘‘for the useful occupations of life.” The college 
enrolls 1,550 students, each of whom is assigned two faculty 
advisers. The average student-faculty ratio is 13:1. Rather than 
complete a traditional major, each Juniata student designs a 
Program of Emphasis tailored to the student’s own goals and 
often crossing departmental lines, www.juniata.edu 
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2008 our students’ scores on this portion of the CLA have 
improved.” 

The University of Findlay’s CLA results for 2007 
through 2009, which indicated that “UF students struggled 
in all areas,” created a “sense of urgency in general 
education assessment.” The university focused on inserting 
problem-based learning into courses: 

“Raising awareness among students, faculty, and 
staff of the CLA and its foci, working to triangulate 
internal data with external measurements such as CLA 
and NSSE, and providing extensive faculty development 
including the formation of a Teaching Academy and a 
mentoring program for junior faculty to promote critical 
thinking and workshops provided opportunities to 
‘make over’ courses with a focus on improving critical 
thinking.” As with Loyola, student achievement at 
University of Findlay improved: “Test results from 2010 
indicated an upward trend in student performance.” 

A direct response to CLA scores in critical thinking 
need not depend on a reaction to disappointing findings. 

At Aurora University (Illinois), very positive findings 
concerning “value-added” on the CLA inspired efforts 
to explain and reinforce these results. A two-day faculty 
workshop in critical thinking led faculty members to 
explore more deeply the kinds of classroom practices that 
could lead to such changes. This was followed by a closer 
comparison of courses in which “instruction characterized 
by the kinds of ill-structured problem-solving on the CLA 
were assessed alongside more traditional teaching methods,” 
producing “an openness to reconsider pedagogical practices 
and opportunities.” 

Aurora’s inquiry into ill-structured problem-solving 
points to one of the most common impacts of the CLA — 
specifically, the introduction of problem-based learning 
and the performance task as elements in the teaching of 




LaGrange College (Georgia) is a four-year liberal arts and 
sciences college affiliated with the United Methodist Church 
The college, an award winner in sustainability, is the oldest 
private institution of higher learning in Georgia and has an 
enrollment of more than 1,000 students, www.lagrange.edu 



critical thinking. These innovations seemed to resonate 
well with faculty members who usually recognized that 
the performance task confronts students with the kind of 
problem-solving challenge that is basic to what almost all 
faculty members do in their own work. Even in disciplines 
that are more structured in their methodology and use 
of evidence, such as mathematics and the sciences, there 
are typically research questions and problems in which 
the evidence is fragmentary and perhaps inconsistent and 
for which there is a need for interpretation and critical 
analysis. Faculty members likely to be suspicious or 
skeptical of standardized testing, including the CLA, often 
are more receptive to the idea of assessing students’ skills 
and development through performance tasks. This has led 
many institutions to use performance tasks (“CLA in the 
Classroom”) and to participate in CAE’s “Performance Task 
Academy,” which will be described in the next section. ■ 
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VI. TEACHING IN A NEW WAY: CLA IN THE CLASSROOM 



“We have become enamored with the performance tasks as a pedagogical 
strategy.... Even those faculty members who have not adopted the use 
of the performance task have been inspired to try out new teaching 
strategies, particularly those geared toward critical thinking.” 

-George Forsythe, President, Westminster Coiiege (Missouri) 



O ne of the attractions of the CLA instrument is that 
its performance task mirrors the activity of critical 
thinking in real-life situations. Students are presented with 
an assignment and some supporting material that does 
not provide a definitive answer to the problem. In writing 
a response the student needs to show an appreciation of 
competing claims and evidence in writing a persuasive 
response to the prompt. Performance tasks combine the 
need for making and breaking arguments, good writing, 
and, of course, critical thinking. At the same time, 
performance tasks also mimic what academics typically do 
in their research by working with ill-structured problems in 
the context of a discipline. Despite the common distinction 
made by colleges and universities between their campuses 
and the “real world,” critical thinking in and beyond the 
academy shares significant features with respect to ill- 
structured problems. 

Although faculty members and others can easily 
recognize the similarity between the CLA and their own 
work, they often also wonder how precisely the CLA 
performance task might have any direct bearing on what 
they teach. A biologist or historian, for example, might 
see the challenge in the CLA results at the institutional 
level but be unsure of how he or she should respond at 
the classroom level. As the report from Morningside 
College stated, “Since CLA is a macro indication. . .the 
most significant challenge has been the adoption of CLA 
pedagogy into the classroom.” Similarly, Texas Lutheran 



University reported “uncertainty regarding where and how 
corrective action should be focused. ... How do we actually 
use the data to drive effective change?” With any data or 
evidence there is always a question of what it might suggest 
for change. In the case of the CLA, making this linkage is 
more difficult because, first, the results are an institutional 
rather than individual measure and, second, because the 
results stand apart from any particular courses, activities, or 
discipline-specific knowledge. 




Loyola University New Orleans (Louisiana), a Jesuit and 
Cathoiic institution of higher education, is grounded in the 
iiberai arts and sciences, whiie aiso offering opportunities for 
professionai studies in undergraduate and seiected graduate 
programs. Through teaching, research, creative activities, and 
service, the university strives to educate the whoie student and 
to benefit the iarger community, www.loyno.edu 
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Lynchburg College (Virginia), founded in 1903, is a private, 
iiberai arts coiiege with an enroiiment of 2,600 students. The 
coiiege offers bacheior’s and master’s degrees, as weii as 
doctorates in physicai therapy and ieadership studies. It is one 
of 40 coiieges featured in Loren Pope’s Colleges that Change 
Lives. The coiiege aiso operates the 450-acre Ciaytor Nature 
Study Center, which inciudes a dark-sky observatory. The 
student-facuity ratio is 12-1, and students reguiariy do hands- 
on research with their professors, www.lynchburg.edu 

Some colleges and universities, as noted in the previous 
section, have responded to this problem with a specific 
focus on teaching critical thinking. However, others 
have seen the idea of a performance task itself as a way 
of revising their work to teach critical thinking, namely 
through the “CLA in the Classroom” initiative. This 
program seeks to apply critical features of the CLA — ill- 
structured problem-based performance tasks — to course 
development and pedagogical practices in the classroom. 

Early in the development of the CLA, thanks in part 
to the urging of CIC/CLA Consortium members, the 
CAE realized that institutions might have some difficulties 
linking CLA results to program and pedagogy. (Eor a more 
detailed description of the development of the CLA in 
the Classroom program, see CIC’s Evidence of Learning: 
Applying the Collegiate Learning Assessment to Improve 
Teaching and Learning.) One response to this issue is to 
introduce performance tasks, and therefore assessment 
of writing and critical thinking, into existing courses in 
a range of fields. CLA in the Classroom complements 
institution-wide assessment by focusing on CLA-style 
assessment of course-level work. In order to do this the 



CAE created the Performance Task Academy, “a series of 
workshops open to faculty/instructors/teachers, teaching 
and learning center staff, as well as others interested in 
pedagogy and curriculum. The central activity of the 
Academy is the construction of discipline and course-based 
performance tasks.” After several years of sponsoring these 
Academies there is now a Performance Task Library of 
tasks constructed by faculty members. (Eor examples of how 
faculty members from Consortium institutions have used 
performance tasks in their classroom teaching, see “Taking 
Teaching to (Performance) Task: Linking Pedagogical and 
Assessment Practices,” in the March/April 2010 issue of 
Change.) 

Eor many faculty members, CLA in the Classroom 
and the Performance Task Academy provide what is 
commonly referred to as “authentic assessment” that 
more directly measures what is being taught and sought. 

By introducing CLA in the Classroom the CAE notes, 

“The traditional modes of assessment of knowledge are 
seen as inadequate because they fail to assess students’ 
capability in the authentic activities of their discipline. The 
authentic assessment movement would instead reflect the 
complex performances that are central to a field of study 
(for example, writing a position paper on an environmental 
issue, investigating a mathematical concept.)” That is, 
while the CLA may give valuable information at the 
institutional level, it does not provide the more specific, 
“authentic” assessment that many faculty members see as 
more directly connected to their work. 

Many colleges in the CIC/CLA Consortium have been 
involved in the CLA in the Classroom and Performance 
Task Academies and have modified courses and pedagogy 
to include performance tasks and testing. Eor example, at 
Charleston Southern University (South Carolina), 21 
faculty members participated in an on-campus Performance 
Task Academy that “resulted in some faculty beginning to 
push CLA measures into various majors, most commonly 
done by adding a Performance Task to class work.” The 
result was a dramatic shift in pedagogical orientation. “Eor 
these faculty members, there is now a greater understanding 
of the connection between assessment and pedagogy.” 

Perhaps the most dramatic example of a shift in 
pedagogical focus created by CLA in the Classroom is the 
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Marian University (Wisconsin) is a Cathoiic university dedicated 
to exceiient teaching and iearning in the Franciscan and liberal 
arts traditions. The educational experience is framed within the 
context of Franciscan values of dignity of the individual, peace 
and justice, reconciliation, and responsible stewardship. Marian 
has a total enrollment of 2,841 students, 71 percent of whom 
are women, www.mahan.edu 




Morningside College (Iowa), founded in 1894, is a private, 
four-year, co-educational institution with a total enrollment of 
approximately 2,000 students. Morningside’s undergraduates 
represent 25 states and 12 countries. The college offers 
five undergraduate degrees: Bachelor of Arts, Bachelor of 
Science, Bachelor of Science in Nursing, Bachelor of Music, 
and Bachelor of Music Education. At the graduate level, 
Morningside confers a Master of Arts in Teaching degree. 
www.morningside.edu 



experience of William Woods University. Faculty members 
were dubious about CLA testing and results, and only 
three (of 65) participated in an information session on 
the CLA. However, in a workshop on performance tasks, 
“nearly all faculty members (60 of 66) attended, and from 
that workshop, 14 faculty members created 18 performance 
tasks. Faculty members have asked for additional workshops 
so they can develop additional tasks.” Following the 
workshop, CLA scores did improve, but faculty members 
nevertheless continued discussion of successes and failures 
with their performance tasks. “Faculty members understand 
that the improved 2010 score may or may not be the result 
of their work but they would like to think so. The next step 
for William Woods is to investigate methodically whether 
the improved score is a fluke or the result of learning.” 

A number of institutions in the Consortium developed 
performance tasks with less broad faculty involvement 
but no less interesting results. For example, at Carlow 
University (Pennsylvania), eight faculty members who 
attended Academies “brought back to campus great energy 
about the performance task concept” with the following 
results: 

“Over the 2009-2010 academic year and fall 2010, 
these faculty members have introduced performance 
tasks in Quantitative Reasoning and College Research 
and Writing, both first-year core curriculum courses; in 
English 100 and College Reading, both developmental 
skills courses; and in an upper level course in Special 

Education Some of the performance tasks have 

taken the form of ‘mini’ assignments that could be done 
within a SO-minute class, while others were larger tasks 
that extended across a semester. These initiatives by 
Carlow faculty members demonstrate that there are many 
possibilities for adaptation and experimentation using 
performance tasks.” 

Similarly, performance tasks can be designed not only 
to produce “authentic” student work and develop critical 
thinking skills but also to promote greater engagement and 
broader communication skills. One particularly interesting 
performance task, “A Referendum on Cell Phone Use 
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While Driving,” was developed for a psychology class at 
the College of St. Scholastica (Minnesota). Using a task 
library consisting of adequate and inadequate online, news 
media, and research documents, small student teams within 
the class each wrote a report explaining and defending a 
pro or con position on a proposed referendum to ban cell 
phone use while driving. These teams then had to put 
their reports together into a final report for a class debate. 

A similar use occurred at Marian University, involving 
a performance task in an honors course. Molecules of 
Emotion, which was taught in the fall of 2010. The task 
was then transformed into a debate team event during 
class. “The result has stimulated wider interest in and 
appreciation of performance tasks.” 

The wide adoption of CLA in the Classroom 
performance tasks is one of the significant, far-reaching 
outcomes of the CIC/CLA Consortium. Since the idea 



of an ill-structured problem is familiar to faculty members 
from all disciplines, the development of performance 
tasks is one way that the CLA can shift approaches to 
the teaching of critical thinking. It has the potential 
to bridge the gap between institution-level assessment 
and outcomes, the CLA as a general measure of critical 
thinking, and critical thinking in the context of teaching 
specific subject matter. It also promotes a pedagogical shift 
to a more student-centered emphasis on problem solving 
and “authentic assessment” of it. President George Lorsythe 
of Westminster College (MO) reflects the sentiment of 
many in the Consortium, “We have become enamored 
with the performance tasks as a pedagogical strategy — 
Even those faculty members who have not adopted the use 
of the performance task have been inspired to try out new 
teaching strategies, particularly those geared toward critical 
thinking.” ■ 
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VII. THE CATALYST: THE CLA’S INFLUENCE ON CHANGES IN 
PROGRAMS AND PEDAGOGY 



“We aspire to a greater level of faculty awareness of the CLA as well as 
data that allow them to target their teaching and student learning strategies 
to improve student learning outcomes.” 

-Kenneth Garren, President, Lynchburg College 



O ne of the words most frequently used in the 

CIC/CLA Consortium campus reports is “catalyst.” 
The CLA results might not be immediately or directly 
connected to change in program or pedagogy, but the 
instrument has sparked consideration of what assessment 
should be and do on a campus, and which academic 
programs or pedagogies need to be revisited. Although 
there is no clear line between direct and indirect impacts 
of the CLA on Consortium institutions, many institutional 
leaders reported that the indirect effects of the CLA 
as “catalyst” were significant. These “catalytic” effects 
involved both programmatic and pedagogical effects as well 
as changes in institutional culture, which will be discussed 
in the next section. 

Sometimes programmatic effects included the CLA’s 
influence on assessment activities on campus. For example, 
Lynchburg College (Virginia) adopted the CLA because it 
had no institution-wide assessments of general education. 
General education outcomes were assessed through course- 
embedded evaluations and “each professor assessed in his or 
her own way.” The CLA was adopted “with the intention 
of improving our institution-level assessment of student 

learning What we did not anticipate at the time was 

a complete revamping of our student learning assessment 
program. The CLA has served as a catalyst for our 
freshman/senior institution-wide assessment. In addition to 
the CLA we now assess freshmen and seniors on writing, 
speaking, reading, and information literacy.” These new 
assessments are then triangulated with other assessments, 
including course-embedded assessments. Thus although 
the CLA did not directly change academic programs and 



pedagogy, it did have a direct impact on the assessment 
program. 

A similar shift occurred at Stephens College (Missouri). 
The CLA was well received and “caused many instructors 
to reevaluate the critical thinking content of their 
courses.” As the college continued its work with the CLA 
it also “initiated a comprehensive assessment planning 
effort for all disciplinary programs and the core liberal 
arts curriculum. The plans include use of a consistent 
framework for assessment across programs, finalizing 
learning and outcomes measures, and designing course- 
embedded teaching strategies and assessments to ensure 




Nebraska Wesleyan University, founded in 1887, is a 
liberal arts institution with small class sizes and a broad- 
based curriculum of 52 majors and three master's level 
programs. Nebraska Wesleyan enrolls approximately 1,600 
undergraduates, 250 graduate students, and 250 adult 
learners, www.nebrwesleyan.edu 
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Notre Dame of Maryland University, founded by the School 
Sisters of Notre Dame, provides a liberal arts education in the 
Catholic tradition. Academics are organized into four schools: 
School of Education, School of Pharmacy, School of Arts and 
Sciences, and School of Nursing. Rich in tradition, Notre Dame 
provides students with opportunities in research, study abroad, 
and service to the global community, www.ndm.edu 

that outcomes are achieved and learning is measured.” The 
introduction of the CLA thus stimulated the development 
of a more wide-ranging and coherent assessment program. 
“In total, our experience with the CLA along with 
program assessment design has created a cadre of faculty 
and administrators with enhanced commitment to critical 
thinking pedagogy.” 

Another way that the CLA served as a catalyst for 
rethinking assessment among Consortium colleges was 
to encourage the triangulation of CLA results with 
other measures. For example, Texas Lutheran University 
administered the National Survey of Student Engagement 
(NSSE) in conjunction with the CLA. The findings 
were instructive, first, because many of the “high-impact 
practices” measured by NSSE were correlated positively 
with student performance on aspects of the CLA. For 
example, a student who reported on NSSE that he was 
required to write more papers tended to score better in 
writing on the CLA. Second, because the CLA results were 
disappointing, the university’s response was to focus on 
high-impact practices implied by the NSSE elements: 

“Over a quarter of the faculty received money this 
past summer to dramatically increase the level of active 



learning and high-impact projects in their classes They 

linked classes, service and community-based learning, 

and required a capstone course While it would be 

disingenuous to state these changes were solely due to 
the CLA, the lower-than-desired CLA scores played an 
important role in driving these changes.” 

An interesting approach to triangulation arose at 
Carlow University in which not administering the CLA 
was a catalyst for reshaping assessment practices. It was felt 
that the case for curricular change had to be built first by 
using home-grown instruments including a local version of 
the performance task administered to juniors, measures of 
critical thinking in the disciplines, and local assessments of 
first-year and senior writing. “It was only at this point that 
an external tool, the CLA, could be introduced. Faculty 
now asked for an external, reliable measure that allowed us 
to see where we stood in relation to other institutions and 
to validate or question what we were seeing in our in-house 
assessment.” Once the data from all these assessments were 
presented, the results were surprisingly similar. “Doubters 
about our internal measures could not also deny the 
CLA results; resistors to external measures could not also 
disabuse the results of the internal assessments.” Setting 
the stage for the use of the CLA in conjunction with other 
measures had a profound effect on the evolution of program 
assessment at Carlow. 

Besides these effects on assessment programs, the 
CLA was also a catalyst for changes in academic programs 
and pedagogy. Similar to Carlow, Springfield College 
(Massachusetts) already had “both a regular program 
review process and an Institutional Assessment Plan.” 

The administration of the CLA and the subsequent 
participation of faculty members in the CLA in the 
Classroom Academy shifted the focus to “using actual 
student course work to assess their learning” — that is, to 
see student work as performances, even if not performance 
tasks. CLA in the Classroom “approaches to pedagogy and 
assessment are natural for our faculty members as most 
are heavily engaged in service learning and implement 
other forms of active and collaborative learning.” Further 
discussion of pedagogy and assessment in these contexts 
suggested “exploring the use of the Association of 
American Colleges and Universities Valid Assessment of 
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Seton Hill University (Pennsylvania), founded in 1885 by the 
Sisters of Charity, is a co-educational, Catholic, liberal arts 
university that offers more than 30 undergraduate programs, 
eight graduate programs and six advanced certifications. The 
university's enrollment of 2,232 represents students from 33 
states and 16 countries, www.setonhill.edu 




Southwestern University (Texas), the first institution of higher 
learning in Texas, is a private, highly selective liberal arts 
institution with 1,300 students. The university features the 
Sarofim School of Fine Arts, living-learning communities, 
and the Paideia Program, in which sophomores participate 
in weekly small reading/discussion groups as a way of 
connecting coursework and out of class experiences. 
www.southwestern.edu 



Learning in Undergraduate Education (VALUE) rubrics 
as part of our assessment of these important aspects of 
our general education.” The CLA served as a catalyst 
to consideration of a different assessment instrument 
consistent with the pedagogy of performance. 

A more specific instance of an indirect effect occurred 
at Dominican University (Illinois). Although the CLA 
has mainly been a spur to thought and discussion across the 
institution, “even apart from the results, the indirect impact 
has been significant. ... The CIC/CLA experience pushed 
us toward integrative learning, and specifically, quantitative 
literacy.” The importance of interpreting quantitative 
information in the CLA performance tasks has led to 
consideration of “some version of quantitative literacy 
across the curriculum. Our experience with the CLA has 
served as a catalyst for this educational advance.” Working 
with the CLA also was a catalyst for a set of conversations 
about ways to incorporate integrative learning throughout 
the curriculum. Both at Dominican and Springfield, the 
CLA was suggestive of alternative pedagogies, assessment 
measures, and other issues that needed study and 
improvement. 

Stories like these have been repeated throughout the 
CIC/CLA Consortium. The administration of the CLA 
prompts a reexamination of programs and practices that, 
although not always directly tied to the CLA instrument 
or its results, is nonetheless a significant change in an 
institution’s approach to student learning. These changes 
can vary from the revamping of a broad, institution-wide 
assessment program, to shifts in programs and pedagogy. 
And often both occurred, as at Aurora University where 
“the intent, at first, was not to directly reform instruction 
but rather to inform development of a coherent, university- 
wide assessment system.” It ended up, not surprisingly, 
doing both: exploring the pedagogy of teaching ill- 
structured problems as suggested by the CLA and 
anticipating a general education model in which the CLA 
becomes integral to the assessment plan. ■ 
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VIII. CHANGING THE CONVERSATION FROM “I/MY” TO “WE/OUR” 



“Already members of our community see assessment of student learning 
as a positive, productive-and essential-part of the learning process. 

This perspective stands in contrast to the viewpoint that assessment is 
an externally imposed mandate with little real meaning.” 

-Rebecca Sherrick, President, Aurora University 



A cademic departments and programs in colleges and 
universities have considerable independence. Faculty 
members often speak of my class or department and what 1 
do as a professor, in a proprietary way. Similarly, observers 
often comment on how institutions of higher education 
have internal “silos” in terms of departments or disciplinary 
areas that do not coordinate their efforts or even know 
what others do. The loosely-coupled or fragmented nature 
of colleges and universities, even relatively smaller ones like 
many CIC members, makes creating or changing shared 
institutional goals and carrying out coordinated, collective 
action very difficult. 

One of the most widespread and potentially positive 
effects of the CLA initiative on Consortium colleges and 
universities was to shift conversations on campus toward 
a more collective perspective. Specifically, in considering 
what the CLA results meant and what appropriate 
responses might be, the conversation almost automatically 
turned to what we were doing in our programs and with our 
students. Instead of assessment being some kind of burden 
required for accreditation, or a puzzling administrative 
fiat, assessment could be seen as important to what faculty 
members — indeed, all members of a college or university 
community — see as their primary aim: improving student 
learning. 

These discussions were not always easy, they were and 
almost never are completely conclusive, and they are and 
will be continuing. Nevertheless, being presented with 
data from the CLA about student results almost always 



precipitated a discussion about what we were or were not 
achieving or doing or going to do, and how our students 
or programs might improve, and so on. The institutional 
focus of the CLA as a measure impelled the discussion to 
collective, institutional considerations. 

Although one might be skeptical that this shift in 
rhetoric is significant — “Talk is cheap” — talk, as has 
already been shown, often led directly or indirectly to 
action. And almost all the institutions could point to the 
ways in which working with the CLA changed the focus 
of the campus conversation to issues of student learning 
and increased coordinated and collective action. The CLA 
has served as a catalyst for reframing the discussion of the 
issue of assessment. CIC’s previous report on the earlier 
phase of the CIC/CLA Consortium, Evidence of Learning, 
describes initial efforts to establish a “culture of assessment” 
particularly at Barton College, Cabrini College, and the 
University of Charleston. 

The conversation often began, not surprisingly, with 
a reaction to whether the news was seen as positive or 
negative. Perhaps the easiest discussion took place when 
the data were generally positive. An institution can both 
take comfort in the results — “we are doing something 
right” — and come to realize that assessment is not 
necessarily something to be feared and avoided. Many of 
the colleges in the Consortium were pleased to find that 
the CLA provided confirmation that they were in fact 
contributing to students’ growth in critical thinking skills 
and that “value-added” could be documented. At the same 
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Springfield College (Massachusetts), founded in 1885, 
offers undergraduate and graduate degree programs in health 
sciences, human and social services, sport management and 
movement studies, education, business, and the arts and 
sciences. More than 5,000 traditional, nontraditional, and 
international students study at its main campus in and at its 
School of Human Services campuses across the country. 

WWW. springfieldcollege. edu 

time, positive results still led to consideration of changes. 

The report of Notre Dame of Maryland University put 
it succinctly: 

“Some of the CLA results validate what we have already 
surmised anecdotally and also confirm what we have seen 
through other measures. . .as well as feedback from our 
graduates. Despite the successes we have achieved through 
this assessment initiative, we also face a series of challenges 

in using the CLA assessment tool The conversations 

should revolve around the implications of CLA results 
as well as how to use the CLA to improve teaching and 
learning.” 

Like Notre Dame of Maryland University, other 
colleges that reported this kind of reassurance almost 
never saw this as an invitation to rest on their laurels. 

For example, Hilbert College reported, “While we have 
been very pleased by the positive value-added results we 
received, CLA results also initiated discussions at various 
levels regarding its implication for the college’s mission 
and goals.” Similarly, Southwestern University reports, 
“Performance on this test tells us that we have very bright 
students who begin and end their college career strong 




Stephens College (Missouri), founded in 1833, is a private 
college and the second-oldest women’s college in the country. 
Stephens offers baccalaureate and master’s degrees and an 
educational experience characterized by intellectual rigor, 
creative expression, and professional practice. Students 
are educated in the liberal arts and professionally prepared 
through internships, community service, and a variety of 
hands-on opportunities, www.stephens.edu 

It lets us know we are doing something right.” Nevertheless, 

“We have definite plans for using the CLA in the future 

One of our main goals is to foster a liberal arts education of 
the highest rank and quality and we will continue to assess 
our general education requirements and other elements of a 
liberal arts education such as critical thinking, writing, and 
problem-solving.” 

The good news about CLA-inspired shifts in 
conversations is that even when results were not entirely 
positive, institutions typically reacted in positive ways. 

For example, at Morningside College, after data showed 
disappointing CLA scores, “a board member asked why 
we were collecting data that might show us in a negative 
light. Another board member responded that this is 
the information we need to improve.” As a result, “the 
CLA has singlehandedly made assessment a front-burner 
topic for faculty and enabled the board of directors to 
better understand student learning.” Similarly, results at 
University of Findlay seemed to show little or no value 
added, and “the campus as a whole was disappointed 
with the results; therefore a number of strategies were 
discussed The implementation of the CLA has led to a 
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sense of urgency in general education assessment.” Finally, 
at Drake University, “senior results. . .have been more 
troubling and served as a notice that Drake must diligently 
analyze additional data sources to determine if the problem 
areas on the CLA are consistent and accurate indicators of 
a problem.” In these and other cases, the consistent theme 
is that the responses to results, negative or positive, lead to 
productive collective conversations and action. 

Beyond reactions to specific results, the CLA has 
shifted conversations on these campuses about assessment 
more generally, especially among faculty members. 

The introduction of the CLA was sometimes met with 
skepticism and even hostility. For example, at Ursuline 
College “reactions to the CLA and assessment of learning 
goals ranged from resentment toward outside controls, 
government intervention in the classroom, and teaching 
to the test.” But “by 2009-2010, the assessment process 
was well established.” The conversation at Ursuline and 
elsewhere evolved as faculty members and administrators 
moved the conversation to the shared task of improving 
student learning. A good example of this shift is at Alaska 
Pacific University: 

“Gradually the faculty appeared to gain confidence that 
assessment efforts were being used primarily to think about 

how students learn and faculty teach The APU faculty 

has grown from being resistant and skeptical about the 
CLA to acknowledging its value as the university evaluates 
what we do well and areas we need to improve.” 

That is not to say that these conversations were 
particularly easy, nor has the process of change always 
been smooth. For example, at Charleston Southern 
University, “the CLA was a tough sell on our campus. 

Some were excited about the results, but others were 
skeptical of the data and the fact that this was not a 
longitudinal study.” Flowever, “With more faculty exposed 
to CLA methodology, through summer meetings and the 
Performance Task Academy, there is greater acceptance of 
the results and use of CLA measures.” Similarly, the report 
of FFastings College (Nebraska) bluntly notes: 

“In general, assessment (including the CLA) is an 
irritating, time-consuming, and absolutely necessary 
process. However, introducing the CLA has helped make 
the documentation of student academic achievement. 



participation, and satisfaction. . .part of our daily routine. 
While we think we already do a pretty good job, we now 

have a much clearer idea of where we can do better In 

addition, comparative assessment data (particularly the 
CLA) are necessary to meet the increasing oversight by the 
board of trustees of the academic program requirements.” 

The collective conversation may not always be smooth, 
and assessment is not easy, but the institutions in the 
Consortium have generally found a way to make these 
conversations productive for faculty and other groups as 
well. 

One major result of these conversations has been the 
growing number of faculty members who have become 
willing to take on responsibility for promoting further 
development of the use of the CLA and other forms of 
assessment. For example, at Stephens College, “experience 
with the CLA along with program assessment design 
has created a cadre of faculty and administrators with 
enhanced commitment to critical-thinking pedagogy.” 
Similarly, at Trinity Christian College (Illinois): 

“As a result of using the CLA. . .we have become more 

in tune to assessment of student learning outcomes 

In addition, working with the CLA has changed the way 
we approach data collection and has encouraged us to 
work in an organized fashion to gather data and design 




Stonehill College (Massachusetts), founded in 1948, is a 
private, Roman Catholic college that offers 80 diverse majors 
and minors in the liberal arts, sciences, and pre-professional 
fields to 2,300 students, www.stonehill.edu 
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Texas Lutheran University provides an education in the 
arts and sciences that is given perspective by the Christian 
faith. This function is carried out through an undergraduate 
curricuium ieading to the bacheior's degree, a diverse 
continuing education program, and a variety of co-curricuiar 
programs, www.tlu.edu 




interventions related to our general education learning 
outcomes.” 

As faculty members and administrators take charge 
of these conversations toward a more institutional focus 
on assessment, the question of “doing” assessment, at 
least among these institutions, is shifting toward how to 
make better use of assessment to improve teaching and 
learning. As the report from LaGrange College (Georgia), 
which started using the CLA in 2008, put it, “The main 
challenge is taking the next step of using CLA results to 
improve teaching.” Taking on this challenge is increasingly 
discussed as part of the responsibility of Consortium 
member colleges and universities. As the report from 
founding Consortium member Allegheny College put it, 
“With the opportunity for direct measures that the CLA 
afforded us, then, it seemed institutionally irresponsible 
not to confirm our success in these areas — or discover 
ways in which our educational program might, in fact, 
be falling short.” Finally, as Rebecca Sherrick, president 
of Aurora University, notes, “Already members of our 
community see assessment of student learning as a positive, 
productive — and essential — part of the learning process. 
This perspective stands in contrast to the viewpoint that 
assessment is an externally imposed mandate with little 
real meaning.” This change in discussions and attitudes 
may be the biggest impact of the CIC/CLA Consortium 
initiative: motivating colleges and universities to improve 
teaching and learning. ■ 



Trinity Christian Coiiege (Illinois) provides a Christ-centered, 
liberal arts education to nearly 1,500 students through more 
than 40 programs of study, various pre-professional programs, 
and the Adult Studies Accelerated Program. Students learn 
through local and global field experience, service projects, 
and study abroad opportunities in countries such as Spain, 
Ecuador, and Nicaragua, www.trnty.edu 
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IX. BUILDING A PROFESSIONAL COMMUNITY ACROSS INSTITUTIONS: 
THE CIC/CLA CONSORTIUM 



“The greatest strength of the Consortium. ..lay in the progression of 
summer meeting topics from necessary logistical matters.. .to substantive 
issues facing higher education and inclusion of outside experts to inform 
our own development.” 

-Mary Hines, President, Cariow University 



I f academic programs and departments are often 
“siloed” — acting independently of one another — the 
institutional autonomy of colleges and universities is far 
more pronounced. Colleges and universities often aspire 
to be distinctive, even unique, in one or all of mission, 
program, and pedagogy. In part this fierce independence 
is a function of the way in which the highly regarded 
system of higher education has developed in the United 
States. The first colleges and universities were private and 
fought for and won constitutional recognition to remain 
so. Even as a now-larger public sector in higher education 
has developed, there is no national ministry of higher 
education, and the two sectors continue to offer students 
an array of options in undergraduate education. 

At the same time — however autonomous colleges 
and universities are and however much they aspire to 
distinguish themselves from one another — they all 
share a common purpose to develop similar knowledge, 
skills, and dispositions in their students. There may be 
differences in mission, program, and pedagogy, but there 
is also considerable overlap. If we were to peruse college 
and university catalogs, we would almost certainly see 
strong similarities in statements of mission and goals. More 
specifically, it is hard to imagine any college or university 
that would not see critical thinking as a desirable outcome 
of an undergraduate education. Whatever the differences 
among institutions, there is also a common, if loosely 
joined, professional community in higher education. 

More than one institution reported that the existence 



of the CIC/CLA Consortium and the support it provided 
were essential to marking effective use of the CLA. 
Franklin Pierce University (New Hampshire) reported 
that, “It is unlikely Franklin Pierce would have engaged 
in this type of general skills assessment as early as we did 
without the CIC Consortium. The Consortium provided 
an opportunity. ..to participate in an assessment strategy we 

otherwise might not have even considered It particularly 

helped us to commiserate about shared challenges and 




University of Charieston (West Virginia) is a private, residential 
university offering professional preparation in a liberal arts 
context. It serves approximately 1,500 students from 34 
states and 35 countries, offering 22 undergraduate majors, 
three master’s degrees, and a doctoral degree in pharmacy. 

UC is a leader in outcomes-based education and assessment 
of student learning, www.ucwv.edu 
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University of Evansviiie (Indiana), established in 1854, 
is a private, liberal arts and sciences-based university 
affiliated with the United Methodist Church. The university’s 
2,961 students represent 45 states and 46 countries. The 
University of Evansville has received national acclaim for its 
study-abroad programs, and the university's British campus, 
Harlaxton College, is located in a 100-room Victorian manor in 
Grantham, England, www.evansville.edu 

possible solutions.” Similarly, Springfield College’s report 
noted that, “1 can’t imagine how we would have made sense 
of our data. . .without the resources the Consortium has 
provided.” 

The CIC/CLA Consortium made success possible 
in several ways: providing technical advice, external 
validation, broader conceptual understanding, and a sense 
of community about issues of assessment. First, and most 
simply, conversations among institutions provided basic 
logistical help and advice. According to Hilbert College, 
a more recent member of the Consortium: “Participating 
in the CIC/CLA Consortium has been the key to the 
successes we have had up to this point. We have benefitted 
greatly from the experiences shared by peer institutions 
that had already faced and often surmounted many of the 
challenges we found ourselves facing, particularly how to 
use the CLA effectively.” 

The College of Saint Benedict and Saint John’s 
University noted that one of the benefits of the annual 
meetings was “sharing suggestions for improving student 
completion of the instrument and trying to improve 
student motivation to take the CLA seriously and do their 



best on the instrument Schools have created a variety 

of clever efforts to deal with the participation/motivation 
problem.” Similar specific assistance was available between 
meetings as well, as Ursuline College reported, “It was easy 
to pick up the phone or email a colleague and ask for advice 
or share concerns.” 

One specific form of logistical and technical 
support emerging from the Consortium was an ongoing 
dialogue between the CAE and the institutions about 
administrative and reporting issues with regard to the 
ongoing development and refinement of the instrument. 

For example. Bethel University (Minnesota) noted, “the 
power in numbers gave us a strong voice in requesting 
changes to the CLA (such as how the reports should look, 
administration issues, etc.). Kudos should go to CAE and 
CIC for listening to us and making the needed adjustments 
to the CLA.” Morningside College’s report noted that “the 
sharing of information and ability to have input into how 
to make the CLA more functional has been very valuable.” 
The ongoing interaction between the CAE and the 
members of the Consortium, including the development of 
many performance task efforts and academies mentioned 
previously, has made the use of the CLA and CLA in the 
Classroom easier and ultimately more productive. 

Beyond technical and logistical support and ideas, the 
Consortium also provided broader but no less valuable 
external validation and better understanding of this 
work that placed campus assessment efforts in a larger 
context. The message that assessment in general and the 
CLA in particular might be valuable was better received 
when it was not only one’s on-campus colleagues who 
were delivering it but colleagues at other institutions 
as well. Many campus leaders adopted the strategy of 
sending a different faculty member to each of the annual 
summer meetings to increase exposure to the CLA and to 
understand better assessment efforts at other Consortium 
institutions. Some institutions would send a fourth team 
member, typically a faculty leader, at the institution’s 
expense to further broaden this exposure. Dominican 
University (Illinois) reported, “As is probably the case at 
most institutions. . .faculty are initially skeptical,” however, 
the “materials provided at the conferences. . .have been very 
helpful in introducing the CLA to other faculty members.” 
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University of Findiay (Ohio), founded in 1882, serves 3,900 
students from 40 states and 25 countries and territories and 
offers nearly 60 majors leading to baccalaureate degrees, 
seven master’s degrees, and doctorates In pharmacy and 
physical therapy. Programs include equestrian studies, animal 
science/pre-veterinary medicine, nuclear medicine technology, 
and environmental, safety, and occupational health 
management, among others, www.findlay.edu 




University of Great Faiis (Montana), founded in 1932, 
is a private. Catholic, liberal arts university that enrolls 
approximately 950 students and has a 12:1 student to 
professor ratio. The university offers 40 undergraduate, eight 
graduate, and four distance learning degrees, www.ugf.edu 



At William Woods University, “The university may not 
have been able to sustain its assessment culture if it had 
not been for the CIC/CLA Consortium. Division chairs 
and an occasional faculty member were able to come back 



to campus and work with other faculty members in their 
respective areas on enhancing curriculum to better serve 
our students. Talking with other faculty members and 
learning how things are accomplished at other institutions 
would invigorate our faculty members and get them excited 
to try something new for the upcoming academic years.” At 
Marian University, faculty skepticism shifted as a result of 
participation in Consortium meetings: 

“Participating in the Consortium has helped build 
momentum around assessment. Presenting CLA results 
did encourage conversation around student learning and 
assessment. This was instrumental in curricular changes 
and also determining if the education at Marian is what is 
needed by students and other constituents.” 

Over several years the conversations at the Consortium 
meetings evolved — away from test administration issues 
and toward more substantive questions — another clear 
benefit and effect of the Consortium. For example, the 
University of Great Falls (Montana) reported that 
later meetings led to a more “sophisticated and nuanced 
understanding of what the CLA test is for and how it can 
become part of a campus culture.” Similarly, Barton College 
reported that after an early focus on technical issues, “later 
meetings focused on what the data could reveal about the 
students, the curriculum, and the quality of the academic 
program.” For Carlow University, the “greatest strength of 
the Consortium. . .lay in the progression of meeting topics 
from necessary logistical matters. . .to substantive issues 
facing higher education and inclusion of outside experts to 
inform our own development.” 

Overall, the Consortium has produced a professional 
community around not just an assessment instrument but 
also the broader issues of how to use assessment to improve 
programs, teaching, and learning. “We have become part 
of a larger community — sharing and learning from one 
another.” Many colleges offered comments like those in the 
report of Seton Fiill University: 

“Consortium participation provided a valuable context 
for gaining a deeper understanding and interpretation of 
results. Members generated commitment, enthusiasm, and 
support for enhancing student learning outcomes.” Several 
colleges expressed the sentiment of Jamestown College that 
this kind of community building should be continued. ■ 
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X. CLOSING THE LOOP: NEW WAYS AND PATHWAYS 



“The CLA is only one tool; we need multiple and varied assessments.” 

-Don Bantz, President, Alaska Pacific University 



I n discussions of assessment it is common to talk about 
“closing the loop.” This phrase is shorthand for moving 
from gathering evidence to making changes on the basis 
of the evidence, and then repeating this cycle to see if 
the changes have led to improvement. Contemporary 
discussions of assessment among colleges and universities 
have noted that there are many efforts being made at 
gathering evidence and accumulating data. It is less clear 
that evidence is being routinely used to guide institutional 
decision-making and direct programmatic changes. (See 
for example the NILOA report, Opening Doors to Faculty 
Involvement in Assessment.) 

Getting from evidence to action is seldom 
straightforward. Often institutions have considerable 
data from many sources but little time or resources for 
assembly and analysis. For example, as Allegheny College 
reported, “one of the biggest obstacles to progress in this 




Upper Iowa University, founded in 1857, is a private university 
providing undergraduate and graduate degree programs 
and leadership development opportunities to nearly 6,800 
students-nationally and internationally, www.uiu.edu 



area, somewhat ironically, is the large volume of data we 
have collected through a large assortment of assessment 
instruments.” Also, as noted previously, one difficulty 
in “closing the loop” is that any piece or set of evidence 
seldom suggests specific changes without considerable 
interpretation and discussion. That discussion often 
requires getting past what the College of St. Scholastica 
labeled “data denial — faculty members who don’t think 
we have a problem because they found fault with the tool 
or sampling methods.” Even when the data, especially 
an indicator such as the CLA, are seen as valid, it is not 
clear what they imply for any specific action. Willamette 
University (Oregon) found “interpreting the results 
challenging, as is identifying actionable changes and 
curricular initiatives.” As Morningside College reported, 
“Since CLA is a macro indication, the question of where 
to focus resources for improvement requires more in- 
depth analysis, which frankly speaking is not always 
available.” It should be noted, again, that such difficulties of 
interpretation and connections to action are not unique to 
CLA; they are complications of any assessment effort. 

Nevertheless, much of the experience of the CIC/CLA 
Consortium suggests, first, that evidence can sometimes 
have the direct effect of suggesting where change is needed 
even if it doesn’t suggest specific changes. A number of 
colleges, as discussed in previous sections, simply made 
changes in the areas that the CLA measures (most notably 
writing and critical thinking) in response to the evidence, 
even though the evidence at hand did not always suggest 
what changes might lead to improvement. At the same 
time, often colleges did focus their efforts on certain 
obvious areas such as making or critiquing an argument. 

Lor other colleges there were more indirect “catalytic” 
effects in which a range of changes going beyond the CLA 
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Ursinus College (Pennsylvania), founded in 1869, is a private, 
undergraduate, liberal arts college that enrolls approximately 
1,750 students with a student-faculty ratio of 12:1. The 
nationally recognized first-year program offers the Common 
Intellectual Experience seminar, housing centers, one-on-one 
faculty advising, and laptops for all students. Students are 
required to complete an Independent Learning Experience 
through either study abroad or major research or an internship 
or student teaching, www.urs'mus.edu 



(to the broader assessment program, in experimenting with 
performance task pedagogy, etc.) were enacted. The “loop” 
may not be tightly closed among these colleges, but the 
ability to refer to evidence is spurring direct and indirect 
action and continuing conversations about assessment. 

There are at least two other ways of building a tighter 
connection between evidence and action found among 
the CIC/CLA Consortium colleges. First, many colleges 
have started to do thoughtful triangulation of CLA with 
other measures. As Don Bantz, president of Alaska Pacific 
University, succinctly puts it, the “CLA is only one tool; we 
need multiple and varied assessments.” This is what many 
Consortium institutions have decided. Like several others, 
Carlow University compared CLA results with some of its 
internal measures, “along with student profile data from the 
HERl College Senior Survey.” The process of triangulation 
often led to the consideration of and experimentation with 
other measures. 

The range of possible combinations of CLA with 
other data is, obviously, huge. Perhaps the most common 
use of the CLA with another assessment instrument is 
in combination with NSSE. A good example is Texas 



Lutheran University, which is administering both CLA and 
NSSE to the same students: “We hope that our analysis of 
these combined data will indicate how to improve student 
achievement and enhance learning outcomes. We hope to 
identify the engagement practices that have the highest 
impact on TLU student results.” Similarly, Westminster 
College (Missouri) reports correlations between NSSE and 
CLA scores “to students through their advisors. The reports 
are targeted. . .with suggestions. . .about specific behaviors 
[students] might consider if they would like to see better 
results in the future.” Einally, a number of institutions 
also mentioned the CIC/NSSE “matching project” of 
connecting CLA and NSSE data as a direction they are or 
will be taking. This project combines NSSE and CLA so 
that institutions can discover more about programmatic 
features and levels of engagement that correlate with gains 
in students’ analytical reasoning, critical thinking, and 
writing skills. 

This in turn suggests a second way that Consortium 
institutions are using CLA to try to close the loop: using 
CLA data to examine curricular areas, specific programs, 
or groups of students. For example, Allegheny College is 
already looking at data across programs and groups, drawing 
some “suggestive inferences from the data: that our seniors 
majoring in the humanities do better at the Performance 
Task than do our seniors majoring in disciplines in the 




Ursuline College (Ohio) offers a holistic Catholic education 
of undergraduate and graduate programs that emphasize the 
whole person and provide personalized attention within a 
liberal arts higher educational environment, www.ursuline.edu 
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natural sciences or social sciences.” A similar finding: “And 
what explains the fact that one student population that 
significantly improves its CLA scores in their first two years 

is our varsity athletes? A subject for further analysis!” 

An unusual strategy was adopted by Westminster 
College (Utah) to generate information useful for program 
level assessment: “We began to focus on seniors in particular 
programs. As many seniors as possible were tested in 
capstone courses for specific majors, distributed widely 
across the college. Each year we rotate programs so that 
over time we can develop a comprehensive view of student 
performance across the college while still giving programs 
information they can use in shaping their own curricula.” 
Southwestern University has tightened its focus even 
further, comparing the “performance of students who 
participated in the Paideia program with those who did not.” 
The program involves sophomores participating in weekly 
small reading/discussion groups “on current issues as a way 
of making intentional connections between coursework and 
out of class experiences.” These examples suggest both the 




Wagner College (New York), founded in 1883, is a private, 
liberal arts college that enrolls about 2,300 students pursuing 
bachelor’s and master’s degrees in business, education, 
nursing, theater, the social and natural sciences, and the 
humanities. Its undergraduates represent 43 states and 15 
foreign countries. The college has become widely known for its 
signature curriculum, the Wagner Plan for the Practical Liberal 
Arts, which combines three levels of interdisciplinary learning 
communities with practical internships and community service. 
www.wagner.edu 



range and potential possibilities for connecting the CLA to 
students’ experience and institutional programs. 

Examining CLA performance by subgroups is central 
to the next phase of CIC’s use of the CLA. CIC’s Pathways 
Project, funded by the Carnegie Corporation of New York, 
focuses specifically on underserved students (particularly 
first-generation and low-income) at nine urban and ten non- 
urban institutions from the current group of 47 institutions, 
plus ten new institutions in urban settings. 

The purpose of the Pathways Project is to assess and 
improve student learning among underserved students 
in urban settings. The CLA will be the primary measure 
of student success. Lor the participating institutions, 
this represents a real opportunity to look at how groups 
of students perform and discover ways to assist them. 

Lor example. Bethel University (Minnesota) sees the 
“challenges” of getting larger samples necessary for analyzing 
subgroups of students. “With these challenges come some 
important opportunities, [but] the ability to see if our Pell 
Grant and first-generation college students are making the 
same gains in writing and critical thinking as our other 
students is important to the mission of our institution.” 
Similarly, Dominican University (Illinois) noted that 
“Because Dominican has long been committed to admitting 
first-generation college students and to creating an 
increasingly diverse student body, we are looking forward to 
the insights provided by this innovative use of the CLA.” 

During the Pathways Project, research is being 
conducted to gain insight into the factors that contribute 
to (or detract from) the academic achievement of 
students at urban institutions, with particular emphasis 
on first-generation and low-income students, which are 
traditionally underrepresented groups. Information on 
effective strategies will be shared with academic institutions 
throughout the CIC membership and more broadly. In 
this way, CIC intends to create both new pathways to 
educational and economic opportunity for students who 
attend urban colleges and universities, many of whom are 
from underserved populations, and to “close the loop” 
by connecting assessment evidence with strategies for 
improvement. The Pathways Project will continue CIC’s 
work with the CLA and with assessment as a means of 
improving undergraduate education. ■ 
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XI. LESSONS LEARNED: IRREVERSIBLE CHANGE, MOMENTUM, 
AND PROFESSIONAL COMMUNITY AND RESPONSIBILITY 



“It is fair to say that at virtually every institution things have changed-in 
some instances a lot and almost always for the better.” 

-Richard Morrill, President, The Teagle Foundation 



T he preceding report of the activities of the colleges 
and universities in the CIC/CLA Consortium merely 
scratches the surface of all the work by colleges that has 
been undertaken during this initiative. Richard Morrill, 
president of the Teagle Foundation, said, “It is fair to say 
that at virtually every institution things have changed — 
in some instances a lot and almost always for the better.” 
The experiences of the institutions offer many lessons, 
particularly about the CLA as an instrument, the process 
of assessment on campuses, the growing community of 
practice around assessment in general and the CLA in 
particular, and the likely future evolution of assessment 
efforts, especially at independent colleges and universities. 
Overall, the initiative has involved a steady expansion — 
one is tempted to say “mushrooming,” — of efforts, changes, 
experimentation, and conversation about ways to use 
outcomes assessment effectively to improve teaching 
and learning. 

The first lesson is that measures and measurement 
matter. The CLA is a powerful and potentially valuable 
instrument. Equally important, having a key measure 
or measures is crucial to focusing discussions of student 
learning at an institution. Although some faculty members 
and others raised questions about whether one can measure 
critical thinking with a single instrument, the vast majority 
of institutions in the Consortium recognized that what the 
CLA measures is central to undergraduate learning. This 
was especially true of the performance task element of the 
CLA and the more general notion of an “ill-structured” 
problem it represented. Every institution wants to improve 
its students’ critical thinking skills, and the CEA provides 
an opportunity to measure those skills in ways that most 



faculty members recognize as being important and 
authentic. 

Similarly, once a measure such as the CEA is used it 
establishes the principle that an institution can and should 
consider it and other kinds of measures and evidence it 
has. What happened over the course of the CIC/CEA 
Consortium’s term was increasing attempts to expand the 
evidentiary base for discussing program and pedagogy. The 
frequent attempts to triangulate CEA results with NSSE 
and to conduct analyses by area or program reflect the fact 
that a variety of measures is available and many analyses 
can be done. Whatever other changes have occurred at the 




Westminster College (Missouri), founded in 1851, is a private, 
residentiai, undergraduate coiiege with a curricuium based on 
the iiberai arts with an emphasis on deveiopmentai experience. 
The coiiege of 1,125 students offers 36 majors, 34 minors, and 
12 pre-professionai programs. The seif-designed major offers 
students an opportunity to design their own major in fieids 
such as advertising, communication, pubiic administration, 
and sports management, www.westminster-mo.edu 



THE COUNCIL OF INDEPENDENT COLLEGES 41 





Westminster College (Utah) is a comprehensive liberal arts 
college with 38 undergraduate majors and 14 graduate 
programs and prepares students for success through active 
and engaged learning, real world experiences, and a vibrant 
campus community. The college currently enrolls 3,100 
students (2,300 undergraduates and 800 graduate students). 
www.westminstercollege.edu 




Willamette University (Oregon) is a private university that 
educates 2,600 students in a residential, undergraduate, 
liberal arts college and professional graduate schools of 
law, management, and education. Willamette is nationally 
recognized for teaching and scholarly work in a community that 
models diversity, sustainability, and service to others, values 
that reflect the university motto, “Not unto ourselves alone are 
we born.” www.witlamette.edu 



colleges in the Consortium, what has changed dramatically 
is a disposition toward use of evidence of student learning 
and analysis of it. Assessment of student learning gains 
much greater legitimacy once institutions have some 
experience using measures and conducting analyses with 
them. 

A second lesson is that the process of measuring 
student learning creates a catalyst for institutional 
change — and it is difficult to imagine going back to the 
way things were. This report catalogs a variety of types of 
curricular and pedagogical change in light of CLA results. 
For some institutions the reaction was immediate and 
focused on those areas — especially critical thinking and 
writing — assessed by the CLA. These and other efforts 
have expanded in a range of ways, from changing specific 
courses and programs, to focusing faculty development 
efforts, to revisiting how assessment was conducted on 
campus more generally. Once these changes are made, 
the logic of assessment and working to “close the loop” 
develops a momentum of its own. This is particularly 
true in terms of campus culture and the shift in the 
conversation to thinking and talking in terms of “we/our.” 
The notion that vue will measure learning and respond to 
evidence represents a significant shift in the culture of most 
colleges and universities, a shift that is moving ahead at the 
institutions in the CIC/CLA Consortium. 

A third lesson is that the CIC/CLA Consortium 
established a professional community of practice that 
supports common measures and practices in assessing and 
improving student learning. For many years, a common 
phrase and injunction in efforts to improve higher 
education has been the need to adopt “best practices.” 
However, the movement from going to a conference 
or workshop in which an interesting “best practice” is 
discussed to going back to campus and putting it into 
practice is usually problematic. Getting attention for an 
idea can be challenging, let alone acquiring the time and 
efforts of individuals actually needed to experiment with a 
new initiative. 

The CIC/CLA Consortium experience provides two 
ways to bridge the gap between a best practice, on the one 
hand, and innovation and campus implementation, on the 
other. First, providing a common measure across a set of 
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similar institutions gives it some measure of credibility. In 
the case of the CLA, apart from its intuitive validity, the 
very fact that a number of institutions were committing to 
experiment with this instrument gave the work some initial 
legitimacy and traction. Second, the existence of ongoing 
meetings of the Consortium provided a real opportunity 
for “best practices” to be developed and disseminated. 
Repeatedly, Consortium colleges and universities 
commented on how interactions with other institutions 
provided advice on everything from logistical challenges of 
testing students to the broadest ideas about curriculum and 
program. The work of the CIC/CLA Consortium provides 
a model of how undergraduate education can become more 
professionalized through shared understandings, measures, 
and practices. 

What is particularly valuable about the Consortium 
in this regard is that it is perfectly consistent with the 
traditional autonomy and diversity of colleges and 
universities. The scope and variety of the work of these 
institutions coupled with the interaction through 
the Consortium offers the possibility for continuing 
experimentation and imitation. Because these are 
independent institutions, they can readily adopt best 
practices as they see them and adapt them to fit their 
individual circumstances. The community of professional 
practice represented by the Consortium shows how greater 
consistency, attention to evidence, transparency, and, 
ultimately, improvement is consistent with institutional 
autonomy and diversity. 

One final lesson that can be drawn from the CIC 
initiative follows from the previous one and brings us back 
to the start of the conversation. The pressure on higher 
education to demonstrate the value of its work, especially 
in light of rising costs, is not likely to fade. However, the 
temptation to use public regulation as a means to spur 
innovation and improvement is also likely to be with 
us, at least for the immediate future. What the CIC/ 

CLA Consortium provides is an example of self-directed, 
voluntary, professional efforts that have and will continue 
to provide a sound response to legitimate public demands 
for assessment, accountability, and improvement. It is 




William Woods University (Missouri), founded in 1870, is an 
independent, co-educational, professions-oriented, liberal 
arts-based institution. Offering undergraduate and graduate 
degrees in a variety of disciplines in both campus and outreach 
settings, WWU enrolls about 3,500 students from 44 states 
and 20 countries. Unique programs include ASL interpreting, 
equestrian studies, juvenile justice, and criminal justice/ 
homeland security. The innovative LEAD (Leading, Educating, 
Achieving ,and Developing) program promotes learning outside 
the classroom and provides an annual tuition reduction. 
www.wiHiamwoods.edu 

extremely unlikely that had the CLA been imposed on 
these CIC institutions by external fiat that we would have 
seen the kind of experimentation, interaction, change, 
and progress that these institutions have achieved over 
the past six years (witness No Child Left Behind). What 
the Consortium has illustrated is that the distinctive 
character of higher education in the United States — its 
mix of diverse, autonomous institutions, including a 
robust private sector engaged in thoughtful cooperation 
and experimentation — does not need external direction 
to continue to improve its work with students. What is 
needed, and found in the CIC/CLA Consortium, is the 
dedicated work of individuals and institutions in higher 
education, particularly in the independent sector, to be 
working constantly together to improve the education of 
their students. ■ 
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CIC/CLA CONSORTIUM PARTICIPANTS (2008-2011) 



Alaska Pacific University#§ 

Allegheny College (PA)*# 

Aurora University (IL) # 

Averett University (VA)# 

Barton College (NC)#§ 

Bethel University (MN)*#§ 

Cabrini College (PA)#§ 

Carlow University (PA)§ 

Charleston Southern University (SC)#§ 

College of Saint Benedict/Saint John’s 
University (MN)*#§ 

College of St. Scholastica (MN) 

Dominican University (1L)§ 

Drake University (lA) 

Franklin Pierce University (NH)# 

Hastings College (NE) 

Hilbert College (NY) 

Illinois College§ 

Indiana Wesleyan University§ 

Jamestown College (ND) 

John Carroll University (OH) 

Juniata College (PA) 

LaGrange College (GA) 

Loyola University New Orleans (LA)# 

Lynchburg College (VA)#§ 

Marian University (Wl)# 

Morningside College (IA)§ 

Nebraska Wesleyan University 

Notre Dame of Maryland University§ 

Seton Hill University (PA)# 



Southwestern University (TX)# 
Springfield College (MA) 

Stephens College (MO) 

Stonehill College (MA)#§ 

Texas Lutheran University#§ 

Trinity Christian College § 

University of Charleston (WV)*#§ 
University of Evansville (IN)# 

University of Lindlay (OH) 

University of Great Lalls (MT)#§ 

Upper Iowa University 
Ursinus College (PA)# 

Ursuline College (OH)# 

Wagner College (NY)# 

Westminster College (MO)#§ 
Westminster College (UT)*# 

William Woods University (MO)# 
Willamette University (OR) 

* 2004-2005 Consortium Participants 

# 2005-2008 Consortium Participants 

§ 2010-2012 Pathways Project Participants 
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